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APPARATUS AND METHODS FOR INCLUDING CODES 
IN AUDIO SIGNALS AND DECODING 
FIELD OF THE INVENTION 
The present invention relates to apparatus and 
5 methods for including codes in audio signals and decoding 
such codes. 

BACKGROUND OF THE INVENTION 
For many years, techniques have been proposed 
for mixing codes with audio signals so that (1) the codes 

10 can be reliably reproduced from the audio signals, while 
(2) the codes are inaudible when the audio signals are 
reproduced as sound. The accomplishment of both 
objectives is essential for practical application. For 
example, broadcasters and producers of broadcast 

15 programs, as well as those who record music for public 
distribution will not tolerate the inclusion of audible 
codes in their programs and recordings. 

Techniques for encoding audio signals have been 
proposed at various times going back at least to U.S. 

20 Patent No. 3,004,104 to Hembrooke issued October 10, 

1961. Heiabrooke showed an encoding method in which audio 
signal energy within a narrow frequency band was 
selectively removed to encode the signal. A problem with 
this technique arises when noise or signal distortion 

25 reintroduces energy into the narrow frequency band so 
that the code is obscured. 

In another method, U.S. Patent No. 3,845,391 to 
Crosby proposed to eliminate a narrow frequency band from 
the audio signal and insert a code therein. This 

30 technique evidently encountered the same problems as 

Hembrooke, as recounted in U.S. Patent No. 4,703,476 to 
Howard which, as indicated thereon, was commonly assigned 
with the Crosby patent. However, the Howard patent 
sought only to improve Crosby's method without departing 

35 from its fundamental approach. 

It has also been proposed to encode binary 
signals by spreading the binary codes into frequencies 
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extending throughout the audio band, a problem with this 
proposed method is that, in the absence of audio signal 
components to mask the code frequencies, they can become 
audible. This method, therefore, relies on the asserted 
5 noiselike character of the codes to suggest that their 
presence will be ignored by listeners. However, in many 
cases this assumption may not be valid, for example, in 
the case of classical music including portions with 
relatively little audio signal content or dviring pauses 
10 in speech. 

A further technique has been suggested in 
which dual tone multifrequency (DTMr) codes are inserted 
in an audio signal. The DTMF Codes are purportedly 
detected based on their frequencies and durations. 

15 However, audio signal components can be mistaken for one 
or both tones of each DTMF code, so that either the 
presence of a code can be missed by the detector or 
signal components can be mistaken for a DTMF code. It is 
noted in addition that each DTMF code includes a tone 

20 common to another DTMF code. Accordingly, a signal 
component corresponding to a tone of a different DTMF 
code can combine with the tone of a DTMF code which is 
simultaneously present in the signal to result in a false 
detection. 

25 OBJECTS OF THE TNVENTIOM 

Accordingly, it is an object of the present 
invention to provide coding and decoding apparatus and 
methods which overcome the disadvantages of the foregoing 
proposed techniques. 
'° It is a further object of the present invention 

to provide coding apparatus and methods for including 
codes with audio signals so that, as sound, the codes are 
inaudible to the human ear but can be detected reliably 
by decoding apparatus. 

A further object of the present invention is to 
provide decoding apparatus and methods for reliably 
recovering codes present in audio signals. 
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SUMMARY OF THE INVENTION 

In accordance with a first aspect of the 
present invention, apparatus and methods for including a 
code having at least one code frequency component with an 
5 audio signal having a plurality of audio signal frequency 
components, comprise the means for and the steps of: 
evaluating an ability of a first set of the plurality of 
audio signal frequency components to mask the at least 
one code frequency component to human hearing to produce 
10 a first masking evaluation; evaluating an ability of a 
second set of the plurality of audio signal frequency 
components differing from the first set thereof to mask 
the at least one code frequency component to human 
hearing to produce a second masking evaluation; assigning 

15 an amplitude to the at least one code frequency component 
based on a selected one of the first and second masking 
evaluations; and including the at least one code 
frequency component with the audio signal. 

In accordance with another aspect of the 

20 present invention, an apparatus for including a code 
having at least one code frequency component with an 
audio signal having a plurality of audio signal frequency 
components, comprises: a digital computer having an input 
for receiving the audio signal, the digital computer 

25 being programmed to evaluate respective abilities of 
first and second sets of the plurality of audio signal 
frequency components to mask the at least one code 
frequency component to human hearing to produce 
respective first and second masking evaluations, the 

30 second set of the plurality of audio signal frequency 
components differing from the first set thereof, the 
digital computer being fiurther programmed to assign an 
amplitude to the at least one code frequency component 
based on a selected one of the first and second masking 

35 evaluations; and means for including the at least one 
code frequency component with the audio signal. 
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In accordance with a further aspect of the 
present invention, apparatus and methods for including a 
code having a plurality of code frequency components with 
an audio signal having a plurality of audio signal 
5 frequency components, the plurality of code frequency 
components including a first code frequency component 
having a first frequency and a second code frequency 
component having a second frecpiency different from the 
first frequency, comprise the means for and the steps of, 

10 respectively: evaluating an ability of at least one of 
the plurality of audio signal frequency components to 
mask a code frequency component having the first 
frequency to human hearing to produce a first respective 
masking evaluation; evaluating an ability of at least one 

15 of the plurality of audio signal frequency components to 
mask a code frequency component having the second 
frecpiency to hiiman hearing to produce a second respective 
masking evaluation; assigning a respective amplitude to 
the first code frequency component based on the first 

20 respective masking evaluation and assigning a respective 
amplitude to the second code frequency component based on 
the second respective masking evaluation; and including 
the plurality of code frequency components with the audio 
signal. 

25 In accordance with yet another aspect of the 

present invention, an apparatus for including a code 
having a plurality of code frequency components with an 
audio signal having a plurality of audio signal frequency 
components, the plurality of code frequency components 

30 including a first code frequency component having a first 
frequency and a second code frequency component having a 
second code frequency different from the first frequency, 
comprises: a digital computer having an input for 
receiving the audio signal, the digital computer being 

35 programmed to evaluate an ability of at least one of the 
plurality of audio signal frequency components to mask a 
code frequency component having the first frequency to 
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human hearing to produce a first respective masking 
evaluation and to evaluate an ability of at least one of 
the plurality of audio signal frequency components to 
mask a code frequency component having the second 
5 frequency to human hearing to produce a second respective 
masking evaluation; the digital computer being further 
programmed to assign a corresponding amplitude to the 
first code frequency component based on the first 
respective masking evaluation and to assign a 

10 corresponding amplitude to the second code frequency 
component based on the second respective masking 
evaluation; and means for including the plurality of code 
frequency components with the audio signal. 

In accordance with a still further aspect of 

15 the present invention, apparatus and methods for 

including a code having at least one code frequency 
component with an audio signal including a plurality of 
audio signal frequency components, comprise the means for 
and the steps of, respectively: evaluating an ability of 

20 at least one of the plurality of audio signal frequency 
components within a first audio signal interval on a time 
scale of the audio signal when reproduced as sound during 
a corresponding first time interval to mask the at least 
one code frequency component to human hearing when 

25 reproduced as sound during a second time interval 

corresponding to a second audio signal interval offset 
from the first audio signal interval to produce a first 
masking evaluation; assigning an amplitude to the at 
least one code frequency component based on the first 

30 masking evaluation; and including the at least one code 
frequency component in a portion of the audio signal 
within the second audio signal interval. 

In accordance with yet still another aspect of 
the present invention, an apparatus for including a code 

35 having at least one code frequency component with an 
audio signal including a plurality of audio signal 
freG[uency components, comprises: a digital computer 
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having an input for receiving the audio signal, the 
digital coiaputer being programmed to evaluate an ability 
of at least one of the plurality of audio signal 
frequency components within a first audio signal interval 
5 on a time scale of the audio signal when reproduced as 
sound during a corresponding first time interval to mask 
the at least one code frequency component to human 
hearing when reproduced as sound during a second time 
interval corresponding to a second audio signal interval 

10 offset from the first audio signal interval, to produce a 
first masking evaluation; the digital computer being 
further programmed to assign an amplitude to the at least 
one code frequency component based on the first masking 
evaluation; and means for including the at least one code 

15 frequency component in a portion of the audio signal 
within the second audio signal interval. 

In accordance with a still further aspect of 
the present invention, apparatus and methods for 
including a code having at least one code frecpiency 

20 component with an audio signal having a plurality of 

audio signal frequency components, comprise the means for 
and the steps of, respectively: producing a first tonal 
signal representing substantially a first single one of 
the plurality of audio signal frequency components; 

25 evaluating an ability of the first single one of the 
plurality of audio signal frequency components to mask 
the at least one code frequency component to human 
hearing based on the first tonal signal to produce a 
first masking evaluation; assigning an amplitude to the 

30 at least one code frequency component based on the first 
masking evaluation; and including the at least one code 
frequency component with the audio signal. 

In accordance with another aspect of the 
present invention, an apparatus for including a code 

35 having at least one code frequency component with an 

audio signal having a plurality of audio signal frequency 
components, comprises: a digital computer having an input 
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for receiving the audio signal, the digital computer 
being programmed to produce a first tonal signal 
representing substantially a first single one of the 
plurality of audio signal frequency components and to 
5 evaluate an ability of the first single one of the 

plurality of audio signal frequency components to mask 
the at least one code frequency component to hxuaan 
hearing based on the first tonal signal to produce a 
first masking evaluation; the digital computer being 

10 further programmed to assign an amplitude to the at least 
one code frequency component based on the first masking 
evaluation; and means for including the at least one code 
frequency component with the audio signal. 

In accordance with yet still another aspect of 

15 the present invention, apparatus and methods for 

detecting a code in an encoded audio signal, the encoded 
audio signal including a plurality of audio frequency 
signal components and at least one code frequency 
component having an amplitude and an audio frequency 

20 selected for masking the code frequency component to 

human hearing by at least one of the plurality of audio 
frequency signal components, comprise the means for and 
the steps of, respectively: establishing an expected code 
amplitude of the at least one code frequency component 

25 based on the encoded audio signal; and detecting the code 
.frecpiency component in the encoded audio signal based on 
the expected code amplitude thereof. 

In accordance with a yet still further aspect 
of the present invention, a programmed digital computer 

30 is provided for detecting a code in an encoded audio 

signal, the encoded audio signal including a plurality of 
audio frequency signal components and at least one code 
frequency component having an amplitude and an audio 
frequency selected for masking the code frequency 

35 component to human hearing by at least one of the 
plurality of audio frequency signal components, the 
digital computer comprising: an input for receiving the 
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encoded audio signal; a processor programmed to establish 
an expected code amplitude of the at least one code 
frequency component based on the encoded audio signal, to 
detect the code frequency component in the encoded audio 
5 signal based on the expected code amplitude and to 
produce a detected code output signal based on the 
detected code frequency component; and an output coupled 
with the processor for providing the detected code output 
signal. 

10 In accordance with another aspect of the 

present invention, apparatus and methods are provided for 
detecting a code in an encoded audio signal, the encoded 
audio signal having a plurality of frequency components 
including a plurality of audio frequency signal 

15 components and at least one code frequency component 
having a predetermined audio frequency and a 
predetermined amplitude for distinguishing the at least 
one code frecpiency component from the plurality of audio 
frequency signal components, comprise the means for and 

20 the steps of, respectively: determining an amplitude of a 
frequency component of the encoded audio signal within a 
first range of audio frequencies including the 
predetermined audio frequency of the at least one code 
frequency component; establishing a noise amplitude for 

25 the first range of audio frequencies; and detecting the 
presence of the at least one code frequency component in 
the first range of audio frequencies based on the 
established noise amplitude thereof and the determined 
amplitude of the frequency component therein. 

30 In accordance with a further aspect of the 

present invention, a digital computer is provided for 
detecting a code in an encoded audio signal, the encoded 
audio signal having a plurality of frequency components 
including a plurality of audio frequency signal 

35 components and at least one code frequency component 
having a predetermined audio frequency and a 
predetermined amplitude for distinguishing the at least 
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one code frequency component from the plurality of audio 
frequency signal components, comprising: an input for 
receiving the encoded audio signal; a processor coupled 
with the input to receive the encoded audio signal and 
5 programmed to determine an amplitude of a freG[uency 
component of the encoded audio signal within a first 
range of audio frequencies including the predetermined 
audio frequency of the at least one code frequency 
component; the processor being further programmed to 

10 establish a noise amplitude for the first range of audio 
frequencies and to detect the presence of the at least 
one code frequency component in the first range of audio 
frequencies based on the established noise amplitude 
thereof and the determined amplitude of the frequency 

15 component therein; the processor being operative to 
produce a code output signal based on the detected 
presence of the at least one code frequency component; 
and an output terminal coupled with the processor to 
provide the code signal thereat. 

20 In accordance with yet a further aspect of the 

present invention, apparatus and methods are provided for 
encoding an audio signal, comprise the means for and the 
steps of, respectively: generating a code comprising a 
plurality of code frequency component sets, each of the 

25 code frecpiency component sets representing a respectively 
different code symbol and including a plurality of 
respectively different code frequency components, the 
code frequency components of the code frequency component 
sets forming component clusters spaced from one another 

30 within the frequency domain, each of the component 
clusters having a respective predetermined frequency 
range and consisting of one frequency component from each 
of the code frequency component sets falling within its 
respective predetermined frequency range, component 

35 clusters which are adjacent within the frequency domain 
being separated by respective frecpiency amounts, the 
predetermined frequency range of each respective 
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component cluster being smaller than the frequency 
amounts separating the respective component cluster from 
its adjacent component clusters; and combining the code 
with the audio signal. 
5 In accordance with yet still another aspect of 

the present invention, a digital computer is provided for 
encoding an audio signal, comprising: an input for 
receiving the audio signal, a processor programmed to 
produce a code comprising a plurality of code frequency 

10 component sets, each of the code frequency component sets 
representing a respectively different code symbol and 
including a plurality of respectively different code 
frequency components, the code' frequency components of 
the code frecpiency component sets forming component 

15 clusters spaced from one another within the frequency 
domain, each of the component clusters having a 
respective predetermined frequency range and consisting 
of one frequency component from each of the code 
frequency component sets falling within its respective 

20 predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, the predetermined 
frequency range of each respective component cluster 
being smaller than the frequency amounts separating the 

25 respective component cluster from its adjacent component 
clusters; and means for combining the code with the audio 
signal. 

The above, and other objects, features and 
advantages of the invention, will be apparent in the 
30 following detailed description of certain advantageous 
embodiments thereof which is to be read in connection 
with the accompanying drawings forming a part hereof, and 
wherein corresponding elements are identified by the same 
reference numerals in the several views of the drawings. 
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BRIEF DESCRIPTION OF THE DRAWIKQg 

Figure 1 is a functional block diagram of an 
encoder in accordance with an aspect of the present 
invention; 

5 Figure 2 is a functional block diagram of a 

digital encoder in accordance with an embodiment of the 
present invention; 

Figure 3 is a block diagram of an encoding 
system for use in encoding audio signals supplied in 
10 analog form; 

Figure 4 provides spectral diagrams for use in 
illustrating frequency compositions of various data 
symbols as encoded by the embodiment of Figure 3; 

Figures 5 and 6 are functional block diagrams 
15 for use in illustrating the operation of the embodiment 
of Figure 3; 

Figures 7A through 70 are flow charts for 
illustrating a software routine employed in the 
embodiment of Figure 3; 
20 Figures 7D and 7E are flow charts for 

illustrating an alternative software routine employed in 
the embodiment of Figure 3 ; 

Figure 7F is a graph showing a linear 
approximation of a single tone masking relationship; 
25 Figiire 8 is a block diagram of an encoder 

employing analog circuitry; 

Figure 9 is a block diagram of a weighting 
factor determination circuit of the embodiment of Figure 
8; 

30 Figure 10 is a functional block diagram of a 

decoder in accordance with certain features of the 
present invention; 

Figure 11 is a block diagram of a decoder in 
accordance with an embodiment of the present invention 
35 employing digital signal processing; 

Figures 12A and 12B are flow charts for use In 
describing the operation of the decoder of Figure 11; 
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Figure 13 is a functional block diagram of a 
decoder in accordance with certain embodiments of the 
present invention; 

Figure 14 is a block diagram of an embodiment 
5 of an analog decoder in accordance with the present 
invention; 

Figure 15 is a block diagram of a component 
detector of the embodiment of Figure 14; and 

Figures 16 and 17 are block diagrams of 
10 apparatus in accordance with an embodiment of the present 
invention incorporated in a system for producing 
estimates of audiences for widely disseminated 
information. 

DETAILED DESCRIPTION OF CERTAIN ADVANTAGEOUS EMBODIMENTS 

i5 Encoding 

The present invention implements techniques for 
including codes in audio signals in order to optimize the 
probability of accurately recovering the information in 
the codes from the signals, while ensuring that the codes 

20 are inaudible to the human ear when the encoded audio is 
reproduced as sound even if the frequencies of the codes 
fall within the audible frequency range. 

With reference first to Figure 1, a functional 
block diagram of an encoder in accordance with an aspect 

25 of the present invention is illustrated therein. An 
audio signal to be encoded is received at an input 
terminal 30. The audio signal may represent, for 
example, a program to be broadcast by radio, the audio 
portion of a television broadcast, or a musical 

30 composition or other kind of audio signal to be recorded 
in some fashion. Moreover, the audio signal may be a 
private communication, such as a telephone transmission, 
or a personal recording of some sort. However, these are 
examples of the applicability of the present invention 

35 and there is no intention to limit its scope by providing 
such examples. 
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As indicated by the functional block 34 in 
Figure 1, the ability of one or more components of the 
received audio signal to mask sounds having frequencies 
corresponding with those of the code f reqpiency component 
5 or components to be added to the audio signal is 

evaluated. Multiple evaluations may be carried out for a 
single code frequency, a separate evaluation for each of 
a plurality of code frequencies may be carried out, 
multiple evaluations for each of a plurality of code 

10 frequencies may be effected, one or more common 

evaluations for multiple code frequencies may be carried 
out or a combination of one or more of the foregoing may 
be implemented. Each evaluation is carried out based on 
the frequency of the one or more code components to be 

15 masked and the frequency or frequencies of the audio 
signal component or components whose masking abilities 
are being evaluated. In addition, if the code component 
and the masking audio component or components do not fall 
within substantially simultaneous signal intervals, such 

20 that they would be reproduced as sound at significantly 
different time intervals, the effects of differences in 
signal intervals between the code component or components 
being masked and the masking program component or 
components are also to be taken into consideration. 

25 Advantageously, in certain embodiments multiple 

evaluations are carried out for each code component by 
separately considering the abilities of different 
portions of the audio signal to mask each code component. 
In one embodiment, the ability of each of a plurality of 

30 substantially single tone audio signal components to mask 
a code component is evaluated based on the frequency of 
the audio signal component, its "amplitude" (as defined 
herein) and timing relevant to the code component, such 
masking being referred to herein as "tonal masking". 

35 The term "amplitude" is used herein to refer to 

any signal value or values which may be employed to 
evaluate masking ability, to select the size of a code 
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component, to detect its presence in a reproduced signal, 
or as otherwise used, including values such as signal 
energy, power, voltage, current, intensity and pressiire, 
whether measured on an absolute or relative basis, and 
5 whether measured on an instantaneous or accumulated 
basis. As appropriate, amplitude may be measured as a 
windowed average, an arithmetic average, by integration, 
as a root -mean-square value, as an accumulation of 
absolute or relative discrete values, or otherwise • 

10 In other embodiments, in addition to tonal 

masking evaluations or in the alternative, the ability of 
audio signal components within a relatively narrow band 
of frequencies sufficiently near a given code component 
to mask the component is evaluated (referred to herein as 

15 "narrow band" masking). In still other embodiments, the 
ability of multiple code components within a relatively 
broad band of frequencies to mask the component is 
evaluated. As necessary or appropriate, the abilities of 
program audio components in signal intervals preceding or 

20 following a given component or components to mask the 
same on a non-simultaneous basis are evaluated. This 
manner of evaluation is particularly useful where audio 
signal components in a given signal interval have 
insufficiently large amplitudes to permit the inclusion 

25 of code components of sufficiently large amplitudes in 

the same signal interval so that they are distinguishable 
from noise. 

Preferably, a combination of two or more tonal 
masking abilities, narrow band masking abilities and 

30 broadband masking abilities (and, as necessary or 

appropriate, non-simultaneous masking abilities) , are 
evaluated for multiple code components. Where code 
components are sufficiently close in frequency, separate 
evaluations need not be carried out for each. 

35 In certain other advantageous embodiments, a 

sliding tonal analysis is carried out instead of separate 
tonal, narrow band and broadband analyses, avoiding the 
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need to classify the program audio as tonal, narrow band 
or broadband. 

Preferably, where a combination of masking 
abilities are evaluated, each evaluation provides a 
5 maximum allowable amplitude for one or more code 

components, so that by comparing all of the evaluations 
that have been carried out and which relate to a given 
component, a maximum amplitude may be selected therefor 
which will ensure that each component will nevertheless 

10 be masked by the audio signal when it is reproduced as 
sound so that all of the components become inaudible to 
human hearing. By maximizing the amplitude of each 
component, the probability of detecting its presence 
based on its amplitude, iis likewise maximized. Of 

15 course, it is not essential that the maximum possible 
amplitude be employed, as it is only necessary when 
decoding to be able to distinguish a sufficiently large 
number of code components from audio signal components 
and other noise. 

20 The results of the evaluations are output as 

indicated at 36 in Figure 1 and made available to a code 
generator 40, Code generation may be carried out in any 
of a variety of different ways. One particularly 
advantageous technique assigns a iinique set of code 

25 frequency components to each of a plurality of data 
states or symbols, so that, during a given signal 
interval, a corresponding data state is represented by 
the presence of its respective set of code frequency 
components. In this manner, interference with code 

30 detection by audio signal components is reduced since, in 
an advantageously high percentage of signal intervals, a 
sufficiently .large number of code components will be 
detectable despite program audio signal interference with 
the detection of other components. Moreover, the process 

35 of implementing the masking evaluations is simplified 
where the frequencies of the code components are known 
before they are generated. 
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Other forms of encoding may also be 
implemented. For example, frequency shift keying (FSK) , 
frequency modulation (FM) , frequency hopping, spread 
spectrum encoding, as well as combinations of the 
5 foregoing can be employed. Still other encoding 

techniques which may be used in practicing the present 
invention will be apparent from its disclosure herein. 

The data to be encoded is received at an input 
42 of the code generator 40 which responds by producing 

10 its unique group of code frequency components and 

assigning an amplitude to each based upon the evaluations 
received from the output 36. The code frequency 
components as thus produced are supplied to a first input 
of a summing circuit 46 which receives the audio signal 

15 to be encoded at a second input. The circuit 46 adds the 
code frequency components to the audio signal and outputs 
an encoded audio signal at an output terminal 50. The 
circuit 46 may be either an analog or digital summing 
circuit, depending on the form of the signals supplied 

20 thereto. The summing function may also be implemented by 
software and, if so, a digital processor used to carry 
out the masking evaluation and to produce the code can 
also be used to sum the code with the audio signal. In 
one embodiment, the code is supplied as time domain data 

25 in digital form which is then summed with time domain 

audio data. In another, the audio signal is converted to 
the frequency domain in digital form and added to the 
code which likewise is represented as digital frequency 
domain data. In most applications, the summed freq[uency 

30 domain data is then converted to time domain data. 

From the following, it will be seen that 
masking evaluation as well as code producing functions 
may be carried out either by digital or analog 
processing, or by combinations of digital and analog 

35 processing. In addition, while the audio signal may be 
received in analog form at the input terminal 30 and 
added to the code components in analog form by the 
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circuit 46 as shown in Figure 1, in the alternative, the 
audio signal may be converted to digital form when it is 
received, added to the code components in digital form 
and output in either digital or analog form. For 
5 example, when the signal is to be recorded on a compact 
disk or on a digital audio tape, it may be output in 
digital form, whereas if it is to be broadcast by 
conventional radio or television broadcasting techniques, 
it may be output in analog form. Various other 

10 combinations of analog and digital processing may also be 
implemented. 

In certain embodiments, the code components of 
only one code symbol at a time are included in the audio 
signal. However, in other embodiments, the components of 

15 multiple code symbols are included simultaneously in the 
audio signal- For example, in certain embodiments the 
components of one symbol occupy one frequency band and 
those of another occupy a second frequency band 
simultaneously. In the alternative, the components of 

20 one symbol can reside in the same band as another or in 
an overlapping band, so long as their components are 
distinguishable, for example, by assigning to 
respectively different frequencies or frequency 
intervals . 

25 An embodiment of a digital encoder is 

illustrated in Figure 2. In this embodiment, an audio 
signal in analog form is received at an input terminal 60 
and converted to digital foirm by an A/D converter 62. 
The digitized audio signal is supplied for masking 

30 evaluation, as indicated functionally by the block 64 

pursuant to which the digitized audio signal is separated 
into frequency components, for example, by Fast Fourier 
Transform (FFT) , wavelet transform, or other time-to- 
frequency domain transformation, or else by digital 

35 filtering. Thereafter, the masking abilities of audio 
signal frequency components within frequency bins of 
interest are evaluated for their tonal masking ability. 
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narrow band masking ability and broadband masking ability 
(and, if necessary or appropriate, for non-simultaneous 
masking ability). Alternatively, the masking abilities 
of audio signal frequency components within frequency 
5 bins of interest are evaluated with a sliding tonal 
analysis. 

Data to be encoded is received at an input 
terminal 68 and, for each data state corresponding to a 
given signal interval, its respective group of code 

10 components is produced, as indicated by the signal 

generation functional block 72, and sulsijected to level 
adjustment, as indicated by the block 76 which is also 
supplied with the relevant masking evaluations. Signal 
generation may be implemented, for example, by means of a 

15 look-up table storing each of the code components as time 
domain data or by interpolation of stored data. The code 
components can either be permanently stored or generated 
upon initialization of the system of Figure 2 and then 
stored in memory, such as in HAH, to be output as 

20 appropriate in response to the data received at terminal 
68. The values of the components may also be computed at 
the time they are generated. 

Level adjustment is carried out for each of the 
code components based upon the relevant masking 

25 evaluations as discussed above, and the code components 
whose amplitude has been adjusted to ensure inaudibility 
are added to the digitized audio signal as indicated by 
the summation symbol 80. Depending on the amount of time 
necessary to carry out the foregoing processes, it may be 

30 desirable to delay the digitized audio signal, as 

indicated at 82 by temporary storage in memory. If the 
audio signal is not delayed, after an FFT and masking 
evaluation have been carried out for a first interval of 
the audio signal, the amplitude adjusted code components 

35 are added to a second interval of the audio signal 
following the first interval. If the audio signal is 
delayed, however, the amplitude adjusted code components 
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can instead be added to the first interval and a 
simultaneous masking evaluation may thus be used. 
Moreover, if the portion of the audio signal during the 
first interval provides a greater masking capability for 
5 a code component added during the second interval than 
the portion of the audio signal during the second 
interval would provide to the code component during the 
same interval, an amplitude may be assigned to the code 
component based on the non-simultaneous masking abilities 

10 of the portion of audio signal within the first interval. 
In this fashion both simultaneous and non-simultaneous 
masking capabilities may be evaluated and an optimal 
amplitude can be assigned to ealch code component based on 
the more advantageous evaluation. 

15 In certain applications, such as in 

broadcasting, or analog recording (as on a conventional 
tape cassette) , the encoded audio signal in digital form 
is converted to analog form by a digital-to-analog 
converter (DAC) 84. However, when the signal is to be 

20 transmitted or recorded in digital form, the DAC 84 may 
be omitted. 

The various functions illustrated in Figure 2 
may be implemented, for example, by a digital signal 
processor or by a personal computer, workstation, 

25 mainframe, or other digital computer. 

Figure 3 is a block diagram of an encoding 
system for use in encoding audio signals supplied in 
analog form, such as in a conventional broadcast studio. 
In the system of Figure 3, a host processor 90 which may 

30 be, for example, a personal computer, supervises the 

selection and generation of information to be encoded for 
inclusion in an analog audio signal received at an input 
terminal 94. The host processor 90 is coupled with a 
keyboard 96 and with a monitor 100, such as a CRT 

35 monitor, so that a user may select a desired message to 
be encoded while choosing from a menu of available 
messages displayed by the monitor 100. A typical message 
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to be encoded in a broadcast audio signal could include 
station or channel identification information, program or 
segment information and/ or a time code. 

Once the desired message has been input to the 
5 host processor 90, the host proceeds to output data 
representing the symbols of the message to a digital 
signal processor (DSP) 104 which proceeds to encode each 
symbol received from the host processor 90 in the form of 
a imigue set of code signal components as described 

10 hereinbelov. In one embodiment, the host processor 
generates a four state data stream, that is, a data 
stream in which each data unit can assume one of four 
distinct data states each representing a unique symbol 
including two synchronizing symbols termed "E" and "S" 

15 herein and two message information symbols "1" and "0" 
each of which represents a respective binary state. It 
will be appreciated that any number of distinct data 
states may be employed. For example, instead of two 
message information symbols, three data states may be 

20 represented by three unique symbols which permits a 
correspondingly larger amount of information to be 
conveyed by a data stream of a given size. 

For example, when the program material 
represents speech, it is advantageous to transmit a 

25 symbol for a relatively longer period of time than in the 
case of program audio having a substantially more 
continuous energy content, in order to allow for the 
natural pauses or gaps present in speech. Accordingly, 
to ensure that information throughput is sufficiently 

30 high in this case, the number of possible message 

information symbols is advantageously increased. For 
symbols representing up to five bits, symbol transmission 
lengths of two, three and four seconds provide 
increasingly greater probabilities of correct decoding. 

35 In some such embodiments, an initial symbol (**E") is 
decoded when (i) the energy in the FFT bins for this 
symbol is greatest, (ii) the average energy minus the 
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Standard deviation of the energy for this symbol is 
greater than the average energy plus the average standard 
deviation of the energy for all other symbols, and (iii) 
the shape of the energy versus time curve for this symbol 
5 has a generally bell shape, peaking at the intersymbol 
temporal boundary. 

In the embodiment of Figure 3, as the DSP 104 
has received the symbols of a given message to be 
encoded, it responds by generating a unique set of code 

10 frequency components for each symbol which it supplies at 
an output 106. With reference also to Figure 4, spectral 
diagrams are provided for each of the four data symbols 
S, E, 0 and 1 of the exemplary data set described above. 
As shown in Figure 4, in this embodiment the symbol S is 

15 represented by a unique group of ten code frequency 
components f^ through f arranged at equal frequency 
intervals in a range extending from a frequency value 
slightly greater than 2 kHz to a frequency value slightly 
less than 3 kHz. The symbol E is represented by a second 

20 unique group of ten code frequency components f^^ through 
^20 arranged in the frequency spectrum at equal intervals 
from a first frequency value slightly greater than 2 kHz 
up to a frequency value slightly less than 3 IcHz, wherein 
each of the code components f through fjo has a unique 

25 frequency value different from all others in the same 
group as well as from all of the frequencies f^ through 
f^Q. The symbol 0 is represented by a further unique 
group of ten code frequency components fji through f3o 
also arranged at equal frequency intervals from a value 

30 slightly greater than 2 kHz up to a value slightly less 
than 3 kHz and each of which has a unique frequency value 
different from all others in the same group as well as 
from all of the frequencies through fjo* Finally, the 
symbol 1 is represented by a further unique group of ten 

35 code frequency components fg^ through f^Q also arranged at 
equal frequency intervals from a value slightly greater 
than 2 kHz to a value slightly less than 3 kHz, such that 



wo 9S/27349 



PCT/US95/03797 



22 

each of the components £3^ through f^Q has a unique 
frequency value different from any of the other frequency 
components f^ through f^^. By using multiple code 
frequency components for each data state so that the code 
5 components of each state are substantially separated from 
one another in frequency, the presence of noise (such as 
non-code audio signal components or other noise) in a 
common detection band with any one code component of a 
given data state is less likely to interfere with 

10 detection of the remaining components of that data state. 

In other embodiments, it is advantageous to 
represent the symbols by multiple frequency components, 
for example ten code tones or frequency components, which 
are not uniformly spaced in frequency, and which do not 

15 have the same offset from symbol to symbol. Avoiding an 
integral relationship between code frequencies for a 
symbol by clustering the tones reduces the effects of 
interfrequency beating and room nulls, that is, locations 
where echoes from room walls interfere with correct 

20 decoding « The following sets of code tone frequency 
components for the four symbols (0, 1, s and E) is 
provided for alleviating the effects of room nulls, where 
fl through f^Q reperesent respective code frequency 
components of each of the four symbols (expressed in 

25 Hertz) : 
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Generally speaking, in the examples provided 
above, the spectral content of the code varies relatively 

15 little when the DSP 104 switches its output from any of 
the data states S, E, 0 and 1 to any other thereof. In 
accordance with one aspect of the present invention in 
certain advantageous embodiments, each code frequency 
component of each symbol is paired with a frequency 

20 component of each of the other data states so that the 
difference therebetween is less than the critical 
bandwidth therefor. For any pair of pure tones, the 
critical bandwidth is a frequency range within which the 
frequency separation between the two tones nay be varied 

25 without substantially increasing loudness. Since the 
frequency separation between adjacent tones in the case 
of each of data states S, £, 0 and 1 is the same, and 
since each tone of each of the data states S, E, O and 1 
is paired with a respective tone of each of the others 

30 thereof so that the difference in frequency therebetween 
is less than the critical bandwidth for that pair, there 
will be substantially no change in loudness upon 
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transition from any of the data states S, E, o and 1 to 
any of the others thereof when they are reproduced as 
sound. Moreover, by minimizing the difference in 
frequency between the code components of each pair, the 
5 relative probabilities of detecting each data state when 
it is received is not substantially affected by the 
frequency characteristics of the transmission path. A 
further benefit of pairing components of different data 
states so that they are relatively close in frequency is 

10 that a masking evaluation carried out for a code 

component of a first data state will be substantially 
accurate for a corresponding component of a next data 
state when switching of states take place. 

Alternatively, in the non-uniform code tone 

15 spacing scheme to minimize the effects of room nulls, it 
will be seen that the frequencies selected for each of 
the code frequency components f through f are clustered 
around a frequency, for example, the frequency components 
for fl, f2 and f3 are located in the vicinity of 1055 Hz, 

20 1180 Hz and 1340 Hz, respectively. Specifically, in this 
exemplary embodiment, the tones are spaced apart by two 
times the FFT resolution, for example, for a resolution 
of 4 Hz, the tones are shown as spaced apart by 8 Hz, and 
are chosen to be in the middle of the frequency range of 

25 an FFT bin. Also, the order of the various frequencies 
which are assigned to the code freg[uency components f ^ 
through f^g for representing the various symbols 0, 1, S 
and E is varied in each cluster. For example, the 
frequencies selected for the components fl, f2 and f3 

30 correspond to the symbols (0, 1, S, E) , (S, E, 0, 1) and 
(E, S, 1, 0), respectively, from lowest to highest 
frequency, that is, (1046.9, 1054.7, 1062.5, 1070.3), 
(1179.7, 1187.5, 1195.3, 1203.1), (1328.1, 1335.9, 
1343.8, 1351.6). A benefit of this scheme is that even 

35 if there is a room null which interferes with correct 
reception of a code component, in general the same tone 
is eliminated from each of the symbols, so it is easier 
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to decode a symbol from the remaining components. In 
contrast, if a room null eliminates a component from one 
symbol but not from another symbol, it is more difficult 
to correctly decode the symbol. 
5 It will be appreciated that, in the 

alternative, either more or less than four separate data 
states or symbols may be employed for encoding* 
Moreover, each data state or symbol may be represented by 
more or less than ten code tones, and while it is 

10 preferable that the same number of tones be used to 

represent each of the data states, it is not essential in 
all applications that the niimber of code tones used to 
represent each data state be the same« Preferably, each 
of the code tones differs in frequency from all of the 

15 other code tones to maximize the probability of 

distinguishing each of the data states upon decoding. 
However, it is not essential in all applications that 
none of the code tone frequencies are shared by two or 
more data states. 

20 Figure 5 is a functional block diagram to which 

reference is made in explaining the encoding operation 
carried out by the embodiment of Figure 3. As noted 
above, the DSP 104 receives data from the host processor 
90 designating the sequence of data states to be output 

25 by the DSP 104 as respective groups of code frequency 
components. Advantageously, the DSP 104 generates a 
look-up table of time domain representations for each of 
the code frequency components f ^ through f4Q which it then 
stores in a RAM thereof, represented by the memory 110 of 

30 Figtire 5. In response to the data received from the host 
processor 90, the DSP 104 generates a respective address 
which it applies to an address input of the memory 110, 
as indicated at 112 in Figure 5, to cause the memory 110 
to output time domain data for each of the ten frequency 

35 components corresponding to the data state to be output 
at that time. 
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With reference also to Figure 6, which is a 
functional block diagram for illustrating certain 
operations carried out by the DSP 104^ the memory 110 
stores a sequence of time-domain values for each of the 
5 frequency components of each of the symbols S, E, 0 and 
1. In this particular embodiment, since the code 
frequency components range from approximately 2 kHz up to 
approximately 3 kHz, a sufficiently large number of time 
domain samples are stored in the memory 110 for each of 

10 the frequency components f^ through f^Q so that they may 
be output at a rate higher than the Nyquist frequency of 
the highest frequency code component. The time domain 
code components are output at an appropriately high rate 
from the memory 110 which stores time-domain components 

15 for each of the code frequency components representing a 
predetermined duration so that (n) time-domain components 
are stored for each of the code frec[uency components f 
through f 40 for (n) time intervals t^ through t„, as shown 
in Figure 6. For example, if the symbol S is to be 

20 encoded during a given signal interval, during the first 
interval t^, the memory 110 outputs the time-domain 
components f ^ through f^o corresponding to that interval, 
as stored in the memory 110, During the next interval, 
the time-domain components f^ through f^Q for the interval 

25 t2 are output by the memory lio. This process continues 
sequentially for the intervals t3 through t^ and back to 
t^ until the diuration of the encoded symbol S has 
expired. 

In certain embodiments, instead of outputting 
30 all ten code components, e.g., fl through flO, during a 
time interval, only those of the code components lying 
within the critical bandwidth of the tones of the audio 
signal are output. This is a generally conservative 
approach to ensuring inaudibility of the code components. 
35 With reference again to Figure 5, the DSP 104 

also serves to adjust the amplitudes of the time-domain 
components output by the memory 110 so that, when the 



wo 95/27349 



PCT/US95/03797 



27 

code frequency components are reproduced as sound, they 
will be masked by components of the audio signal in which 
they have been included such that they are inaudible to 
human hearing. Consequently, the DSP 104 is also 
5 supplied with the audio signal received at the input 
terminal 94 after appropriate filtering and analog-to- 
digital conversion. More specifically, the encoder of 
Figure 3 includes an analog band pass filter 120 which 
serves to substantially remove audio signal frequency 

10 components outside of a band of interest for evaluating 
the masking ability of the received audio signal which in 
the present embodiment extends from approximately 1.5 kHz 
to approximately 3.2 kHz. The filter 120 also serves to 
remove high frequency components of the audio signal 

15 which may cause aliasing when the signal is subsequently 
digitized by an analog-to-digital converter (A/D) 124 
operating at a sufficiently high sampling rate. 

As indicated in Figure 3, the digitized audio 
signal is supplied by the A/D 124 to DSP 104 where, as 

20 indicated at 130 in Figure 5, the program audio signal 

undergoes frequency range separation. In this particular 
embodiment, frequency range separation is carried out as 
a Fast Fourier Transform (FFT) which is performed 
periodically with or without temporal overlap to produce 

25 successive frequency bins each having a predetermined 
frequency width, other techniques are available for 
segregating the frequency components of the audio 
signals, such as a wavelet transform, discrete Walsh 
Hadamard transform, discrete Hadamard transform, discrete 

30 cosine transform, as well as various digital filtering 
techniques. 

Once the DSP 104 has separated the frequency 
components of the digitized audio signal into the 
successive frequency bins, as mentioned above, it then 
35 proceeds to evaluate the ability of various frequency 
components present in the audio signal to mask the 
various code components output by the memory 110 and to 
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produce respective amplitude adjustment factors which 
serve to adjust the amplitudes of the various code 
frequency components such that they will be masked by the 
program audio when reproduced as sound so that they will 
5 be inaudible to human hearing « These processes are 
represented by the block 134 in Figure 5. 

For audio signal components that are 
substantially simultaneous with the code frequency 
components they are to mask (but which precede the code 

10 f recpiency components by a short period of time) , the 
masking ability of the program audio components is 
evaluated on a tonal basis, as well as on a narrow band 
masking basis and on a broadband masking basis, as 
described below. For each code frequency component which 

15 is output at a given time by the memory 110, a tonal 

masking ability is evaluated for each of a plurality of 
audio signal frequency components based on the energy 
level in each of the respective bins in which these 
components fall as well as on the frequency relationship 

20 of each bin to the respective code frequency component. 
The evaluation in each case (tonal, narrow band and 
broadband) may take the form of an amplitude adjustment 
factor or other measure enabling a code component 
amplitude to be assigned so that the code component is 

25 masked by the audio signal. Alternatively, the 
evaluation may be a sliding tonal analysis. 

In the case of narrow band masking, in this 
embodiment for each respective code frequency component 
the energy content of frequency components below a 

30 predetermined level within a predetermined frequency band 
including the respective code frequency component is 
evaluated to derive a separate masking ability 
evaluation. In certain implementations narrow band 
masking capability is measured based on the energy 

35 content of those audio signal frequency components below 
the average bin energy level within the predetermined 
frequency band. In this implementation, the energy 
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levels of the components below the energy levels of the 
components below the average bin energy (as a component 
threshold) are siimmed to produce a narrow band energy 
level in response to which a corresponding narrow band 
5 masking evaluation for the respective code component is 
identified. A different narrow band energy level may 
instead be produced by selecting a component threshold 
other than the average energy level. Moreover, in still 
other embodiments, the average energy level of all audio 

10 signal components within the predetermined frequency band 
instead is used as the narrow band energy level for 
assigning a narrow band masking evaluation to the 
respective code component. In still further embodiments, 
the total energy content of audio signal components 

15 within the predetermined frequency band instead is used, 
while in other embodiments a minimum component level 
within the predetermined frequency band is used for this 
purpose. 

Finally, in certain implementations the 

20 broadband energy content of the audio signal is 

determined to evaluate the ability of the audio signal to 
mask the respective code frequency component on a 
broadband masking basis. In this embodiment, the 
broadband masking evaluation is based on the minimum 

25 narrow band energy level found in the course of the 

narrow band masking evaluations described above. That 
is, if four separate predetermined frequency bands have 
been investigated in the course of evaluating narrow band 
masking as described above, and broadband noise is taken 

30 to include the minimum narrow band energy level among all 
four predetermined frequency bands (however determined) , 
then this minimum narrow band energy level is multiplied 
by a factor ecjual to the ratio of the range of 
frequencies spanned by all four narrow bands to the 

35 bandwidth of the predetermined frecpiency band having the 
minimum narrow band energy level. The resulting product 
indicates a permissible overall code power level. If the 
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overall permissible code power level is designated P, and 
the code includes ten code components, each is then 
assigned an amplitude adjustment factor to yield a 
component power level which is 10 dB less than P. In the 
5 alternative, broadband noise is calculated for a 

predetermined, relatively wide band encompassing the code 
components by selecting one of the techniques discussed 
above for assessing the narrow band energy level but 
Instead using the audio signal components throughout the 

10 predetermined, relatively wide band. Once the broadband 
noise has been determined in the selected manner, a 
corresponding broadband masking evaluation is assigned to 
each respective code component.' 

The amplitude adjust factor for each code 

15 frequency component is then selected based upon that one 
of the tonal, narrow band and broadband masking 
evaluations yielding the highest permissible level for 
the respective component. This maximizes the probability 
that each respective code frecpiency component will be 

20 distinguishable from non-audio signal noise while at the 
same time ensuring that the respective code frequency 
component will be masked so that it is inaudible to human 
hearing. 

The amplitude adjust factors are selected for 
25 each of tonal, narrow band and broadband masking based on 
the following factors and circumstances. In the case of 
tonal masking, the factors are assigned on the basis of 
the frequencies of the audio signal components whose 
masking abilities are being evaluated and the frequency 
30 or frecjuencies of the code components to be masked. 

Moreover, a given audio signal over any selected interval 
provides the ability to mask a given code component 
within the same interval (i.e., simultaneous masking) at 
a maximum level greater than that at which the same audio 
35 signal over the selected interval is able to mask the 
same code component occurring before or after the 
selected interval (i.e., non-simultaneous masking) • The 
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conditions under which the encoded audio signal will be 
heard by an audience or other listening group ^ as 
appropriate, preferably are also taken into 
consideration. For example, if television audio is to be 
5 encoded, the distorting effects of a typical listening 
environment are preferably taken into consideration, 
since in such environments certain frequencies are 
attenuated more than others. Receiving and reproduction 
equipment (such as graphic equalizers) can cause similar 

10 effects. Environmental and equipment related effects can 
be compensated by selecting sufficiently low amplitude 
adjust factors to ensure masking under anticipated 
conditions. 

In certain embodiments only one of tonal, 

15 narrow band or broadband masking capabilities are 

evaluated. In other embodiments two of such different 
types of masking capabilities are evaluated, and in still 
others all three are employed. 

In certain embodiments, a sliding tonal 

20 analysis is employed to evaluate the masking capability 
of the audio signal. A sliding tonal analysis generally 
satisfies the masking rules for narrow band noise, 
broadband noise and single tones without requiring audio 
signal classification. In the sliding tonal analysis, 

25 the audio signal is regarded as a set of discrete tones, 
each being centered in a respective PFT frequency bin. 
Generally, the sliding tonal analysis first computes the 
power of the audio signal in each FFT bin. Then, for 
each code tone, the masking effects of the discrete tones 

30 of the audio signal in each FFT bin separated in 
frequency from such code tone by no more than the 
critical bandwidth of the audio tone are evaluated based 
on the audio signal power in each such bin using the 
masking relationships for single tone masking. The 

35 masking effects of all of the relevant discrete tones of 
the audio signal are summed for each code tone, then 
adjusted for the number of tones within the critical 
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bandwidth of the audio signal tones and the complexity of 
the audio signal. As explained below, in certain 
embodiments, the complexity of the program material is 
empirically based on the ratio of the power in the 
5 relevant tones of the audio signal and the root sum of 
squares power in such audio signal tones. The complexity 
serves to account for the fact that narrow band noise and 
broadband noise each provide much better masking effects 
than are obtained from a simple summation of the tones 
10 used to model narrow band and broadband noise. 

In certain embodiments which employ a sliding 
tonal analysis, a predetermined number of samples of the 
audio signal first undergo a Idrge FFT, which provides 
high resolution but requires longer processing time. 

15 Then, successive portions of the predetermined number of 
samples undergo a relatively smaller FFT, which is faster 
but provides less resolution. The amplitude factors 
found from the large FFT are merged with those found from 
the smaller FFTs, which generally corresponds to time 

20 weighting the higher "frequency accuracy" large FFT by 
the higher "time accuracy" of the smaller FFT. 

In the embodiment of Figure 5, once an 
appropriate amplitude adjust factor has been selected for 
each of the code frequency components output by the 

25 memory 110, the DSP 104 adjusts the amplitude of each 

code frequency component accordingly, as indicated by the 
functional block "amplitude adjust" 114. In other 
embodiments, each code freqniency component is initially 
generated so that its amplitude conforms to its 

30 respective adjust factor. With reference also to Figure 
6, the amplitude adjust operation of the DSP 104 in this 
embodiment multiplies the ten selected ones of the time 
domain code frequency components values f through f^Q for 
the current time interval t^ through t„ by a respective 

35 amplitude adjust factor Gj^^ through G^^^q and then the DSP 
104 proceeds to add the amplitude adjusted time domain 
components to produce a composite code signal which it 
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supplies at its output 106* With reference to Figures 3 
and 5, the composite code signal is converted to analog 
fonn by a digital-to-analog converter (DAC) 140 and 
supplied thereby to a first input of a summing circuit 
5 142. The summing circuit 142 receives the audio signal 
from the input terminal 94 at a second input and adds the 
composite analog code signal to the analog audio signal 
to supply an encoded audio signal at an output 146 
thereof • 

10 In radio broadcasting applications, the encoded 

audio signal modulates a carrier wave and is broadcast 
over the air. In NTSC television broadcasting 
applications, the encoded audio signal frequency 
modulates a subcarrier and is mixed with a composite 

15 video signal so that the combined signal is used to 

modulate a broadcast carrier for over-the-air broadcast. 
The radio and television signals, of course, may also be 
transmitted by cable (for example, conventional or fiber 
optic cable), satellite or otherwise. In other 

20 applications, the encoded audio can be recorded either 
for distribution in recorded form or for subsequent 
broadcast or other wide dissemination. Encoded audio may 
also be employed in point-to-point transmissions. 
Various other applications, and transmission and 

25 recording techniques will be apparent. 

Figures 7 A through 7C provide flow charts for 
illustrating a software routine carried out by the DSP 
104 for implementing the evaluation of tonal, narrow band 
and broadband masking fiinctions thereof described above. 

30 Figure 7A illustrates a main loop of the software program 
of the DSP 104. The program is initiated by a command 
from the host processor 90 (step 150), whereupon the DSP 
104 initializes its hardware registers (step 152) and 
then proceeds in step 154 to compute unweighted time 

35 domain code component data as illustrated in Figure 6 
which it then stores in memory to be read out as needed 
to generate the time domain code components, as mentioned 
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hereinabove. In the alternative, this step may be 
omitted if the code components are stored permanently in 
a ROM or other nonvolatile storage. It is also possible 
to calculate the code component data when required, 
5 although this adds to the processing load. Another 

alternative is to produce unweighted code components in 
analog form and then adjust the amplitudes of the analog 
components by means of weighting factors produced by a 
digital processor. 

10 Once the time domain data has been computed and 

stored, in step 156 the DSP 104 communicates a request to 
the host processor 90 for a next message to be encoded. 
The message is a string of characters, integers, or other 
set of data symbols uniquely identifying the code 

15 component groups to be output by the DSP 104 in an order 
which is predetermined by the message. In other 
embodiments, the host, knowing the output data rate of 
the DSP, determines on its own when to supply a next 
message to the DSP by setting an appropriate timer and 

20 supplying the message upon a time-out condition. In a 

further alternative embodiment, a decoder is coupled with 
the output of the DSP 104 to receive the output code 
components in order to decode the same and feed back the 
message to the host processor as output by the DSP so 

25 that the host can determine when to supply a further 

message to the DSP 104. In still other embodiments, the 
functions of the host processor 90 and the DSP 104 are 
carried out by a single processor. 

Once the next message has been received from 

30 the host processor, pursuant to step 156, the DSP 

proceeds to generate the code components for each symbol 
of the message in order and to supply the combined, 
weighted code frequency components at its output 106. 
This process is represented by a loop identified by the 

35 tag 160 in Figure 7A. 

Upon entering the loop symbolized by the tag 
160, the DSP 104 enables timer interrupts l and 2 and 
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then enters a "compute weighting factors" subroutine 162 
which will be described in connection with the flow 
charts of Figures 7B and 7C. With reference first to 
Figure 7B, upon entering the compute weighting factors 
5 subroutine 162 the DSP first determines whether a 
sufficient number of audio signal samples have been 
stored to permit a high-resolution FFT to be carried out 
in order to analyze the spectral content of the audio 
signal during a most recent predetermined audio signal 

10 interval, as indicated by step 163. Upon start up, a 
sufficient number of audio signal samples must first be 
accumulated to carry out the FFT. However, if an 
overlapping FFT is employed, during subsequent passes 
through the loop correspondingly fewer data samples need 

15 be stored before the next FFT is carried out. 

As will be seen from Figure 7B, the DSP remains 
in a tight loop at the step 163 awaiting the necessary 
sample accumulation. Upon each timer interrupt 1, the 
A/D 124 provides a new digitized sample of the program 

20 audio signal which is accumulated in a data buffer of the 
DSP 104, as indicated by the subroutine 164 in Figure 7A. 

Returning to Figure 7B, once a sufficiently 
large number of sample data have been accumulated by the 
DSP, processing continues in a step 168 wherein the 

25 above-mentioned high resolution FFT is carried out on the 
audio signal data samples of the most recent audio signal 
interval. Thereafter, as indicated by a tag 170, a 
respective weighting factor or amplitude adjust factor is 
computed for each of the ten code frequency components in 

30 the symbol currently being encoded. In a step 172, that 
one of the frequency bins produced by the high resolution 
FFT (step 168) which provides the ability to mask the 
highest level of the respective code component on a 
single tone basis (the "dominant tonal") is determined in 

35 the manner discussed above. 

With reference also to Figure 7C, in a step 
176, the weighting factor for the dominant tonal is 
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determined and retained for comparison with relative 
masking abilities provided by narrow band and broadbsmd 
masking and, if found to be the most effective masker, is 
used as the weighting factor for setting the amplitude of 
5 the current code frequency component. In a subsecpient 
step 180, an evaluation of narrow band and broadband 
masking capabilities is carried out in the manner 
described above. Thereafter, in a step 182, it is 
determined whether narrow band masking provides the best 

10 ability to mask the respective code component and if so, 
in a step 184, the weighting factor is updated based on 
narrow band masking. In a subsequent step 186, it is 
determined whether broadband masking provides the best 
ability to mask the respective code frequency component 

15 and, if so, in a step 190, the weighting factor for the 
respective code frequency component is adjusted based on 
broadband masking. Then, in step 192 it is determined 
whether weighting factors have been selected for each of 
the code frequency components to be output presently to 

20 represent the current symbol and, if not, the loop is re- 
initiated to select a weighting factor for the next code 
frequency component. If, however, the weighting factors 
for all components have been selected, then the 
subroutine is terminated as indicated in step 194. 

25 Upon the occurrence of timer interrupt 2, 

processing continues to a subroutine 200 wherein the 
functions illustrated in Figure 6 above are carried out. 
That is, in the subroutine 200 the weighting factors 
calculated during the subroutine 162 are used to multiply 

30 the respective time domain values of the current symbol 
to be output and then the weighted time domain code 
component values are added and output as a weighted, 
composite code signal to the DAC 140. Each code symbol 
is output for a predetermined period of time upon the 

35 expiration of which processing returns to the step 156 
from the step 202. 
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Figures 7D and 7E show flowcharts illustrating 
an implementation of the sliding tonal analysis technique 
for evaluating the masking effects of an audio signal. 
At step 702, variables are initialized such as the size 
5 in samples of a large FFT and a smaller FFT, the nuxaber 
of smaller FFTs per large FFT and the number of code 
tones per symbol, for example, 2048, 256, 8 and 10, 
respectively. 

At steps 704-708, a number of samples 

10 corresponding to a large FFT is analyzed. At step 704, 
audio signal samples are obtained. At step 706, the 
power of the program material in each FFT bin is 
obtained. At step 708, the permissible code tone power 
in each corresponding FFT bin, accounting for the effects 

15 of all of the relevant audio signal tones on that bin, is 
obtained, for each of the tones. The flowchart of Figure 
7E shows step 708 in more detail. 

At steps 710-712, a number of samples 
corresponding to a smaller FFT is analyzed, in similar 

20 fashion to steps 706-708 for a large FFT. At step 714, 
the permissible code powers found from the large FFT in 
step 708 and the smaller FFT in step 712 are merged for 
the portion of the samples which have undergone a smaller 
FFT. At step 716, the code tones are mixed with the 

25 audio signal to form encoded audio, and at step 718, the 
encoded audio is output to DAC 140. At step 720, it is 
decided whether to repeat steps 710-718, that is, whether 
there are portions of audio signal samples which have 
undergone a large FFT but not a smaller FFT. Then, at 

30 step 722, if there are any more audio samples, a next 
nximber of samples corresponding to a large FFT is 
analyzed. 

Figiire 7E provides detail for steps 708 and 
712, computing the permissible code power in each FFT 
35 bin. Generally, this procedure models the audio signal 
as comprising a set of tones (see examples below) , 
computes the masking effect of each audio signal tone on 
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each code tone, stims the masking effects and adjusts for 
the density of code tones and complexity of the audio 
signal. 

At step 752, the band of interest is 
5 determined. For example, let the bandwidth used for 

encoding be 800 Hz - 3200 Hz, and the sampling frequency 
be 44100 samples/sec. The starting bin begins at 800 Hz, 
and the ending bin ends at 3200 Hz. 

At step 754, the masking effect of each 

10 relevant audio signal tone on each code in this bin is 
determined using the masking curve for a single tone, and 
compensating for the non-zero audio signal FFT bin width 
by determining (1) a first masking value based on the 
assumption that all of the audio signal power is at the 

15 upper end of the bin, and (2) a second masking value 
based on the assumption that all of the audio signal 
power is at the lower end of the bin, and then choosing 
that one of the first and second masking values which is 
smaller. 

20 Figure 7F shows an approximation of a single 

tone masking curve for an audio signal tone at a 
frequency of fPGM which is about 2200 Hz in this example, 
following Zwislocki, J. J., "Masking: Experimental and 
Theoretical Aspects of Simultaneous, Forward, Backward 
25 and central Masking", 1978, in Zwicker et al., ed., 
Psvchoa coustics! Facts and Models , pages 283-316, 
Springer-Veriag, New York. The width of the critical 
band (CB) is defined by Zwislocki as: 

critical band = 0.002 * fpcM^*^ + 
30 With the following definitions, and letting "masker" be 
the audio signal tone, 

BRKPOINT = 0.3 /+/- 0.3 critical bands/ 

PEAKFAC = 0.025119 / -16 db from masker/ 
BEATFAC = 0.002512 / -26 db from masker/ 
35 mNEG = -2.40 / -24 db per critical band/ 

mPOS = -0.70 / -7 db per critical band/ 

of s= code frequency 
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mf « masker frequency 
cband = critical band around 
then the masking factor, mf actor, can be computed as 
follows: 
5 brkpt = cband * BRKPOINT 

if on negative slope of curve of Figure 7F, 

mf actor = PEAKFAC * 10**(mNEG* mf-brkpt-cf) /cband) 
if on flat part of curve of Figvure 7F, 
mf actor = BEATFAC 
lb if on positive slope of curve of Figure 7F, 

mf actor « PEAKFAC * 10**{mP0S * cf-brkpt-mf) /cband) 
Specifically, a first mf actor is computed based 
on the assumption that all of the audio signal power is 
at the lower end of its bin, then a second mf actor is 
15 computed assuming that all of the audio signal power is 
at the upper end of its bin, and the smaller of the first 
and second mfactors is chosen as the masking value 
provided by that audio signal tone for the selected code 
tone. At step 754, this processing is performed for each 
20 relevant audio signal tone for each code tone. 

At step 756, each code tone is adjusted by each 
of the masking factors corresponding to the audio signal 
tones. In this embodiment, the masking factor is 
multiplied by the audio signal power in the relevant bin. 
25 At step 758, the result of multiplying the 

masking factors by the audio signal power is stunmed for 
each bin, to provide an allowable power for each code 
tone. 

At step 760, the allowable code tone powers are 
30 adjusted for the number of code tones within a critical 
bandwidth on either side of the code tone being 
evaluated, and for the complexity of the audio signal. 
The number of code tones within the critical band, CTSUM, 
is counted. The adjustment factor, ADJFAC, is given by: 
35 ADJFAC = GLOBAL * (PSUM/PRSS) / CTSUM 

Where GLOBAL is a derating factor accounting for encoder 
inaccuracy due to time delays in FFT performance. 
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(PSUM/PRSS)^-5 is an empirical complexity correction 
factor, and l/CTSUM represents simply dividing the audio 
signal power over all the code tones it is to mask. PSUM 
is the sum of the masking tone power levels assigned to 
5 the masking of the code tone whose ADJFAC is being 
determined. The root sum of squares power (PRSS) is 
given by 

PRSS = SQRT(Z (Pi^)), i = FFT bins in band 
i 

10 For example, assuming a total masking tone power in a 

band equally spread among one, two and three tones, then 





no. 

tones 


tone power 


PSUH 


PRSS 




1 


10 


1 * 10 = 10 


10 


15 


2 


5, 5 


2 * 5 = 10 


SQRT<2*52) s 7.07 




3 


3.3, 3,3, 3.3 


3 * 3.3 = 10 


SQRT(3*3.32) = 5.77 



Thus, PRSS measures masking power peakiness (increasing 
values) or spread-out-ness (decreasing values) of the 

20 program material. 

At step 762 of Figure 7E, it is determined 
whether there are any more bins in the band of interest, 
and if so, they are processed as described above. 

Examples of masking calculations will now be 

25 provided. An audio signal symbol at 0 dB is assimed, so 
that the values provided are the maximum code tone powers 
relative to the audio signal power. Four cases are 
provided: a single 2500 Hz tone; three tones at 2000, 
2500 and 3000 Hz; narrow band noise modelled as 75 tones 

30 within the critical band centered at 2600, that is, 75 
tones equally spaced at 5 Hz in the 2415 to 2785 Hz 
range; and broadband noise modelled as 351 tones equally 
spaced at 5 Hz in the 1750 to 3250 Hz range. For each 
case, a sliding tonal analysis (STA) calculated result is 

35 compared with the calculated result of selecting the best 
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of the single tone, narrow band noise and wideband noise 
analyses • 







SINGLE TONE 


HULTZPLE 
TONES 


NARROW BAND 
NOISE 


BROADBAND 

NOISE 


5 


code 
tone 

(H2) 


STA 
(dB) 


OP 3 
(dB) 


(dB) 


BES7 
OF 3 
(dB) 


(dB) 


OF 3 
(dB) 


(dB) 


B£ST 
OF 3 
(dB) 




1976 


-50 


-49 


-28 


-30 


-19 


NA 


14 


12 




2070 


-45 


—45 


-22 


-32 


-14 


NA 


13 


12 


10 


2163 


-40 


-39 


-29 


-25 


-9 


NA 


13 


12 




2257 


-34 


-33 


-28 


-28 


-3 


NA 


12 


12 




2351 


-28 


-27 


-20 


-28 


I 


NA 


12 


12 




2444 


-34 


-34 


-23 


-33 


2 


7 


13 


12 




2538 


-34 


-34. 


-24 


-34 


3 


7 


13 


12 


15 


2632 


-24 


-24 


-18 


-24 


5 


7 


14 


12 




2726 


-26 


-26 


-21 


-26 


5 


7 


14 


12 




2819 


-27 


-27 


-22 


-27 


6 


NA 


15 





For example, in the sliding tonal analysis 
20 (STA) for the single tone case, the masking tone is 2500 
Hz, corresponding to a critical bandwidth of 0.002 * 
2500^*^ + 100 = 350 Hz. The breakpoints for the curve of 
Figure 7F are at 2500 ± 0.3*350 or 2395 and 2605 Hz. The 
code freqniency of 1976 is seen to be on the negative 
25 slope portion of the curve of Figure 7F, so the masking 
factor is: 

mfactor = 0.025119 * 10"2-4 * (2500-105-1976) /350 
= 3.364 * 10"^ 
= -44.7 dB 

30 There are three codes tones within the critical band of 
1976 Hz, so the masking power is split among them: 
3.364 * 10"5 / 3 = -49.5 dB 
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This result is rounded to the -50 dB shown in the upper 
left of the sample calculations table. 

In the "Best of 3" analysis, tonal masking is 
calculated according to the single tone method explained 
5 above in conjunction with Figure 7F, 

In the "Best of 3" analysis, narrow band noise 
masking is calculated by first computing the average 
power across a critical band centered on the frequency of 
the code tone of interest. Tonals with power greater 

10 than the average power are not considered as part of the 
noise and are removed. The summation of the remaining 
power is the narrow band noise power. The maximum 
allowable code tone power is -6 dB of the narrow band 
noise power for all code tones within a critical 

15 bandwidth of the code tone of/ interest. 

In the "Best of 3" analysis, broadband noise 
masking is calculated by calculating the narrow band 
noise power for critical bands centered at 2000, 2280, 
2600 and 2970 Hz. The minimum resulting narrow band 

20 noise power is multiplied by the ratio of the total 

bandwidth to the appropriate critical bandwidth to find 
the broadband noise power. For example, if the 2600 Hz 
centered band having a 370 Hz critical bandwidth is the 
minimum, its narrow band noise power is multiplied by 

25 1322 Hz / 370 Hz « 3.57 to produce the broadband noise 
power. The allowed code tone power is -3 dB of the 
broadband noise power. When there are ten code tones, 
the maximum power allowed for each is 10 dB less, or -13 
dB of the broadband noise power. 

30 The sliding tonal analysis calculations are 

seen to generally correspond to the "Best of 3" 
calculations, indicating that the sliding tonal analysis 
is a robust method. Additionally, the results provided 
by the sliding tonal analysis in the case of multiple 

35 tones are better, that is, allow larger code tone powers, 
. than in the "Best of 3" analysis, indicating that the 
sliding tonal analysis is suitable even for cases which 
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do not fit neatly into one of the "Best of 3" 
calculations • 

Referring now to Figure 8, an eiabodiment of an 
encoder which employs analog circuitry is shown in block 
5 form therein. The analog encoder receives an audio 

signal in analog form at an input terminal 210 from which 
the audio signal is supplied as an input to N component 
generator circuits 220^^ through 220,, each of which 
generates a respective code component through C,|. For 
10 simplicity and clarity only component generator circuits 
220^^ and 220,1 are shown in Figure 8, In, order to 
controllably generate the code components of a respective 
data symbol to be included in the audio signal to foinn an 
encoded audio signal, each of the component generator 
15 circuits is supplied with a respective data input 

terminal 222^ through 222^ which serves as an enabling 
input for its respective component generator circuit. 
Each symbol is encoded as a subset of the code components 
Cj^ through C,, by selectably applying an enabling signal to 
20 certain ones of the component generator circuits 220^^ 
through 220||. The generated code component is 
corresponding with each data symbol are supplied as 
inputs to a summing circuit 22 6 which receives the input 
audio signal from the input terminal 210 at a further 
input, and serves to add the code components to the input 
audio signal to produce the encoded audio signal which it 
supplies at an output thereof. 

Each of the component generator circuits is 
similar in construction and includes a respective 
weighting factor determination circuit 230^ through 230,|, 
a respective signal generator 232^ through 232^, and a 
respective switching circuit 234^ through 234^* Each of 
the signal generators 232^ through 232,, produces a 
respectively different code component frequency and 
supplies the generated component to the respective 
switching circuit 234^ through 234,,, each of which has a 
second input coupled to ground and an output coupled with 
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an input of a respective one of multiplying circuits 2363^ 
through 236^, In response to receipt of an enabling input 
at its respective data input terminal 222^ through 222^, 
each of the switching circuits 234^^ through 234^ responds 
5 by coupling the output of its respective signal generator 
232 2 through 232j, to the input of the corresponding one of 
multiplying circuits 236^ through 236^. However, in the 
absence of an enabling signal at the data input, each 
switching circuit 234^ through 234^ couples its output to 
10 the grounded input so that the output of the 

corresponding multiplier 236^ through 2 3 6j, is at a zero 
level. 

Each weighting factor deteraination circuit 
230^ through 230^ iserves to evaluate the ability of 

15 frequency components of the audio signal within a 

corresponding frequency . band thereof to mask the code 
component produced by the corresponding generator 2322 
232„ to produce a weighting factor which it supplies as 
an input to the corresponding multiplying circuit 236^ 

20 through 236^ in order to adjust the amplitude of the 
corresponding code component to ensure that it will be 
masked by the portion of the audio signal which has been 
evaluated by the weighting factor determination circuit. 
With reference also to Figure 9, the construction of each 

25 of the weighting factor determination circuits 230^ 

through 230^, indicated as an exemplary circuit 230, is 
illustrated in block form. The circuit 230 includes a 
masking filter 240 which receives the audio signal at an 
input thereof and serves to separate the portion of the 

30 audio signal which is to be used to produce a weighting 
factor to be supplied to the respective one of the 

* 

multipliers 236^ through 236,^. The characteristics of the 
masking filter, moreover, are selected to weight the 
amplitudes of the audio signal frequency components 
35 according to their relative abilities to mask the 
respective code component. 
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The portion of the audio signal selected by the 
masking filter 240 is supplied to an absolute value 
circuit 242 which produces an output representing an 
absolute value of a portion of the signal within the 
5 frequency band passed by the masking filter 240. The 
output of the absolute value circuit 242 is supplied as 
an input to a scaling amplifier 244 having a gain 
selected to produce an output signal which, when 
multiplied by the output of the corresponding switch 234^^ 

10 through 234„, will produce a code component at the output 
of the corresponding multiplier 236^^ through 236,, which 
will ensure that the multiplied code component will be 
masked by the selected portion of the audio signal passed 
by the masking filter 240 when the encoded audio signal 

15 is reproduced as sound. Each weighting factor 

determination circuit 230^^ through 230jj, therefore, 
produces a signal representing an evaluation of the 
ability of the selected portion of the audio signal to 
mask the corresponding code component, 

20 In other embodiments of analog encoders in 

accordance with the present invention, multiple weighting 
factor determination circuits are supplied for each code 
component generator, and each of the multiple weighting 
factor determination circuits corresponding to a given 

25 code component evaluates the ability of a different 
portion of the audio signal to mask that particular 
component when the encoded audio signal is reproduced as 
sound. For example, a plurality of such weighting factor 
determination circuits may be supplied each of which 
' 30 evaluates the ability of a pozrtion of the audio signal 
within a relatively narrow frequency band (such that 
audio signal energy within such band will in all 
likelihood consist of a single frequency component) to 
mask the respective code component when the encoded audio 

35 is reproduced as sound. A further weighting factor 

determination circuit may also be supplied for the same 
respective code component for evaluating the ability of 
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audio signal energy within a critical band having the 
code component frequency as a center frequency to mask 
the code component when the encoded audio signal is 
reproduced as sound. 
5 In addition, although the various elements of the Figures 
8 and 9 embodiment are implemented by analog circuits , it 
will be appreciated that the same functions carried out 
by such analog circuits may also be implemented, in whole 
or in part, by digital circuitry • 

10 Decoding 

Decoders and decoding methods which are 
especially adapted for decoding audio signals encoded by 
the inventive techniques disclosed hereinabove, as well 
as generally for decoding codes included in audio signals 

15 such that the codes may be distinguished therefrom based 
on amplitude, will now be described. In accordance with 
certain features of the present invention, and with 
reference to the fiinctional block diagram of Figtire 10, 
the presence of one or more code components in an encoded 

20 audio signal is detected by establishing an expected 
amplitude or amplitudes for the one or more code 
components based on either or both of the audio signal 
level and a non-audio signal noise level as indicated by 
the functional block 250. One or more signals 

25 representing such expected amplitude or amplitudes are 
supplied, as at 252 in Figure 10, for determining the 
presence of the code component by detecting a signal 
corresponding to the expected amplitude or amplitudes as 
indicated by the functional block 254. Decoders in 

30 accordance with the present invention are particularly 
well adapted for detecting the presence of code 
components which are masked by other components of the 
audio signal since the amplitude relationship between the 
code components and the other audio signal components is, 

35 to some extent, predetermined. 

Figure 11 is a block diagram of a decoder in 
accordance with an embodiment of the present invention 
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which employs digital signal processing for extracting 
codes from encoded audio signals received by the decoder 
in analog form. The decoder of Figure 11 has an input 
terminal 260 for receiving the encoded analog audio 

5 signal which may be, for example, a signal picked up by a 
microphone and including television or radio broadcasts 
reproduced as sound by a receiver, or else such encoded 
analog audio signals provided in the form of electrical 
signals directly from such a receiver. Such encoded 

0 analog audio may also be produced by reproducing a sound 
recording such as a compact disk or tape cassette. 
Analog conditioning circuits 262 are coupled with the 
input 260 to receive the encoded analog audio and serve 
to carry out signal amplification, automatic gain control 

5 and anti-aliasing low-pass filtering prior to analog-to- 
digital conversion. In addition, the analog conditioning 
circuits 262 serve to carry out a bandpass filtering 
operation to ensure that the signals output thereby are 
limited to a range of frequencies in which the code 

0 components can appear. The analog conditioning circuits 
262 output the processed analog audio signals to an 
analog-to-digital converter (A/D) 263 which converts the 
received signals to digital form and supplies the same to 
a digital signal processor (DSP) 266 which processes the 

5 digitized analog signals to detect the presence of code 
components and determines the code symbols they 
represent. The digital signal processor 266 is coupled 
with a memory 270 (comprising both program and data, 
storage memories) and with input /output (I/O) circuits 

0 272 to receive external commands (for example, a command 
to initiate decoding or a command to output stored codes) 
and to output decoded messages. 

The operation of the digital decoder of Figure 
11 to decode audio signals encoded by means of the 

5 apparatus of Figure 3 will now be described. The analog 
conditioning circuits 262 serve to bandpass filter the 
encoded audio signals with a passband extending from 
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approximately 1.5 IcHz to 3.1 kHz and the DSP 266 samples 
the filtered analog signals at an appropriately high 
rate* The digitized audio signal is then separated by 
the DSP 266 into frequency component ranges or bins by 
5 FFT processing. More specifically, an overlapping, 

windowed FFT is carried out on a predetermined number of 
the most recent data points, so that a new FFT is 
performed periodically upon receipt of a sufficient 
number of new samples. The data are weighted as 

10 discussed below and the FFT is performed to produce a 
predetermined number of frequency bins. each having a 
predetermined width. The energy B(i) of each frequency 
bin in a range encompassing the code component 
frequencies is computed by the DSP 266. 

15 A noise level estimate is carried out around 

each bin in which a code component can occur. 
Accordingly, where the decoder of Figure 11 is used to 
decode signals encoded by the embodiment of Figure 3, 
there are 40 frequency bins within which a code component 

20 can appear. For each such frequency bin a noise level is 
estimated as follows- First, an average energy E(j) in 
the frec[uency bins within a window extending in frequency 
above and below the particular frequency bin of interest 
j (that is, the bin in which the code component can 

25 appear) is computed in accordance with the following 
relationship: 

where i = (j-w) (j+w) and w represents the extent of 
the window above and below the bin of interest in numbers 
of bins. Then a noise level NS{j) in the frequency bin j 
30 is estimated in accordance with the following formula: 

NS{j)^(EBn(i))/(E6(i)) 

where Bn(i) equals B(i) (the energy level in bin i) if 
B(i) < E(j) and B(i) equals zero otherwise, and S(i) 
equals 1 if B(i) < E(j) and S(i) equals zero otherwise. 
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That is, noise components are assumed to include those 
components having a level less than the average energy 
level within the particular window surrounding the bin of 
interest, and thus include audio signal components which 
5 fall below such average energy level. 

Once the noise level for the bin of interest 
has been estimated, a signal-to-noise ratio for that bin 
SNR{j) is estimated by dividing the energy level B(j) in 
the bin of interest by the estimated noise level NS(j), 
10 The values of SNR(j) are employed both to detect the 

presence and timing of synchronization symbols as well as 
the states of data symbols, as discussed below. Various 
techniques may be employed to eliminate audio signal 
components from consideration as code components on a 
15 statistical basis. For example, it can be assumed that 
the bin having the highest signal to noise ratio includes 
an audio signal component. Another possibility is to 
exclude those bins having an SNR(j) above a predetermined 
value. Yet another possibility is to eliminate from 
20 consideration those bins having the highest and/ or the 
lowest SNR( j) • 

When used to detect the presence of codes in 
audio signals encoded by means of the apparatus of Figure 
3, the apparatus of Figure 11 accumulates data indicating 
the presence of code components in each of the bins of 
interest repeatedly for at least a major portion of the 
predetermined interval in which a code symbol can be 
found. Accordingly, the foregoing process is repeated 
multiple times and component presence data is accumulated 
for each bin of interest over that time frame. 
Techniques for establishing appropriate detection time 
frames based on the use of synchronization codes will be 
discussed in greater detail hereinbelow. Once the DSP 
266 has accumulated such data for the relevant time 
frame, it then determines which of the possible code 
signals was present in the audio signal in the manner 
discussed below. The DSP 266 then stores the detected 
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code symbol In the memory 270 together with a time stamp 
for identifying the time at which the symbol was detected 
based on an internal clock signal of the DSP. 
Thereafter, in response to an appropriate command to the 
5 DSP 266 received via the I/O circuit 272, the DSP causes 
the memory 270 to output the stored code symbols and time 
stamps via the I/O circuits 272. 

The flow charts of Figures 12A and 12B 
Illustrate the sequence of operations carried out by the 

10 DSP 266 in decoding a symbol encoded in the analog audio 
signal received at the input terminal 260. With 
reference first to Figure 12A, upon initiation of the 
decoding process, the DSP 266 enters a main program loop 
at a step 450 in which it sets a flag SYNCH so that the 

15 DSP 266 first commences an operation to detect the 

presence of the sync symbols E and S in the input audio 
signal in a predetermined message order. Once step 450 
is carried out the DSP 266 calls a sub-routine DET, which 
is illustrated in the flow chart of Figure 12B to search 

20 for the presence of code components representing the sync 
symbols in the audio signal. 

Referring to Figure 12B, in a step 454, the DSP 
gathers and stores samples of the input audio signal 
repeatedly until a sufficient number has been stored for 

25 carrying out the FFT described above. Once this has been 
accomplished, the stored data are subjected to a 
weighting function, such as a cosine squared weighting 
function, Kaiser-Bessel function, Gaussian (Poisson) 
function, Hanning function or other appropriate weighting 

30 function, as indicated by the step 456, for windowing 
the data. However, where the code components are 
sufficiently distinct, weighting is not required. The 
windowed data is then subjected to an overlapped FFT, as 
indicated by the step 460. 

35 Once the FFT has been completed, in a step 462 

the SYNCH flag is tested to see if it is set (in which 
case a sync symbol is expected) or reset (in which case a 
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data bit symbol is expected) • Since initially the DSP 
sets the SYNCH flag to detect the presence of code 
components representing sync symbols, the program 
progresses to a step 466 wherein the frequency domain 
5 data obtained by means of the FFT of step 460 is 

evaluated to determine whether such data indicates the 
presence of components representing an E sync symbol or 
an S sync symbol. 

For the purpose of detecting the presence and 

10 timing of synchronization symbols, first the sum of the 
values of SNR{j) for each possible sync symbol and data 
symbol is determined. At a given time during the process 
of detecting synchronization symbols, a particular symbol 
will be expected. As a first step in detecting the 

15 expected symbol, it is determined whether the sum of its 
corresponding values SNR(j) is greater than any of the 
others. If so, then a detection threshold is established 
based upon the noise levels in the frequency bins which 
can contain code components. That is, since, at any 

20 given time, only one code symbol is included in the 
encoded audio signal, only one quarter of the bins of 
interest will contain code components. The remaining 
three quarters will contain noise, that is, program audio 
components and/ or other extraneous energy. The detection 

25 threshold is produced as an average of the values SNR(j} 
for all forty of the frequency bins of interest, but can 
be adjusted by a multiplication factor to account for the 
effects of euQAbient noise and/or to compensate for an 
observed error rate. 

30 When the detection threshold has thus been 

established, the sum of the values SNR(j) of the expected 
synchronization symbol is compared against the detection 
threshold to determine whether or not it is greater than 
the threshold. If so, a valid detection of the expected 

35 synchronization symbol is noted. Once this has been 

accomplished, as indicated by the step 470, the program 
returns to the main processing loop of Figure 12A at a 
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step 472 where it is determined (as explained 
hereinbelow) whether a pattern of the decoded data 
satisfies predetermined qualifying criteria. If not, 
processing returns to the step 450 to recommence a search 

5 for the presence of a sync symbol in the audio signal, 
but if such criteria are met, it is determined whether 
the expected sync pattern (that is, the expected sequence 
of symbols E and S) has been received in full and 
detected, as indicated by the step 474. 

0 However, after the first pass through the sub- 

routine DET, insufficient data will have been gathered to 
determine if the pattern satisfies the qualifying 
criteria, so that from the step 474, processing returns 
to the sub-routine DET to carry out a further FFT and 

5 evaluation for the presence of a sync symbol • Once the 
sub-routine DET has been carried out a predetermined 
number of times, when processing returns to step 472 the 
DSP determines whether the accumulated data satisfies the 
qualifying criteria for a sync pattern. 

D That is, once DET has been carried out such 

predetermined number of times, a corresponding nvimber of 
evaluations have been carried out in the step 466 of the 
sub-routine DET. The number of times an "E" symbol was 
found is used in one embodiment as a measure of the 

5 amount of "E** symbol energy during the corresponding time 
period. However, other measures of **E" symbol energy 
(such as the total of *'E" bin SNR's which exceed the 
average bin energy) may instead be used. After the sub- 
routine DET is again called and a further evaluation is 

5 carried out in the step 466^ in the step 472 this most 
recent evaluation is added to those accumulated during 
the predetermined interval and the oldest evaluation 
among those previously accumulated is discarded. This 
process continues during multiple passes through the DET 

5 sub-routine and in the step 472 a peak in the symbol 
energy is sought. If such a peak is not found, this 
leads to a determination that a sync pattern has not been 
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encountered, so that processing returns from the step 472 
to the step 450 to set the SYNCH flag once again and 
recommence the search for a sync pattern. 

If, however, such a maximum of the "E" signal 
5 energy has been found, the evaluation process carried out 
in the step 472 after the sub-routine DET 452 continues 
each time using the same number of evaluations from the 
step 466, but discarding the oldest evaluation and adding 
the newest, so that a sliding data window is employed for 

10 this purpose. As this process continues, after a 
predetermined number of passes in the step 472 it is 
determined whether a cross-over from the "E" symbol to 
the "S" has occurred • This is determined in one 
embodiment as the point where the total of "S" bin SNR's 

15 resulting from the step 466 within the sliding window 

first exceeds the total of "E" bin SNR's during the same 
interval. Once such a cross-over point has been found, 
processing continues in the manner described above to 
se£u:ch for a maximum of the "S" symbol energy which is 

20 indicated by the greatest number of "S" detections within 
the sliding data window. If such a maximim is not found 
or else the maximum does not occur within an expected 
time frame after the maximum of the "E" symbol energy, 
processing proceeds from the step 472 back to the step 

25 450 to recommence the search for a sync pattern. 

If the foregoing criteria are satisfied, the 
presence of a sync pattern is declared in the step 474 
and processing continues in the step 480 to determine the 
expected bit intervals based on the **E" and **S*' symbol 

30 energy maxima and the detected cross-over point. Instead 
of the foregoing process for detecting the presence of 
the sync pattern, other strategies may be adopted. In a 
fiirther embodiment, a sync pattern which does not satisfy 
criteria such as those described above but which 

35 approximates a qualifying pattern (that is, the detected 
pattern is not clearly non-qualifying) , a determination 
whether the sync pattern has been detected may be 
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postponed pending further analysis based upon evaluations 
carried out (as explained herein- below) to determine the 
presence of data bits in expected data intervals 
following the potential sync pattern. Based on the 
5 totality of the detected data, that is, both during the 
suspected sync pattern interval and during the suspected . 
bit intervals, a retrospective qualification of the 
possible sync pattern may be carried out. 

Returning to the flow chart of Fig. 12A, once 
10 the sync pattern has been qualified, in the step 480^ as 
noted above, the bit timing is determined based upon the 
two maxima and the cross-over point. That is^ these 
values are averaged to determine the expected start and 
end points of each subsequent data bit interval. Once 
15 this has been accomplished, in a step 482 the SYNCH flag 
is reset to indicate that the DSP will then search for 
the presence of either possible bit state. Then the sub- 
routine DET 452 is again called and, with reference to 
Figure 12B as well, the sub-routine is carried out in the 
20 same fashion as described above until the step 462 

wherein the state of the SYNCH flag indicates that a bit 
state should be determined and processing proceeds then 
to a step 486. In the step 486, the DSP searches for the 
presence of code components indicating either a zero bit 
25 state or a one bit state in the manner described 
hereinabove. 

Once this has been accomplished, at the step 
470 processing returns to the main processing loop of 
Figure 12A in a step 490 where it is determined whether 
30 sufficient data has been received to determine the bit 
state. To do so, multiple passes must be made through 
the sub-routine 452, so that after the first pass, 
processing returns to the sub-routine DET 452 to carry 
out a further evaluation based on a new FFT. Once the 
35 sub-routine 452 has been carried out a predetennined 

number of times, in the step 486 the data thus gathered 
is evaluated to determine whether the received data 
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indicates either a zero state, a one state or an 
indeterminate state (which could be resolved with the use 
of parity data). That is, the total of the "0" bin SNR's 
is compared to the total of the "1" bin SNR's. Whichever 
5 is greater determines the data state, and if they are 
equal, the data state is indeterminate. In the 
alternative, if the "0" bin and "1" bin SNR totals are 
not equal but rather are close, an indeterminate data 
state may be declared. Also, if a greater number of data 
10 symbols are employed, that symbol for which the highest 
SNR summation is found is determined to. be the received 
symbol. 

When the processing again returns to the step 
490, the determination of the bit state is detected and 

15 processing continues to a step 492 wherein the DSP stores 
data in the memory 270 indicating the state of the 
respective bit for assembling a word having a 
predetermined number of symbols represented by the 
encoded components in the received audio signal. 

20 Thereafter, in a step 496 it is determined whether the 

received data has provided all of the bits of the encoded 
word or message. If not, processing returns to the DET 
sub-routine 452 to determine the bit state of the next 
expected message symbol. However, if in the step 496 it 

25 is determined that the last symbol of the message has 

been received, processing returns to the step 450 to set 
the SYNCH flag to search for the presence of a new 
message by detecting the presence of its sync symbols as 
represented by the code components of the encoded audio 

30 signal. 

With reference to Figure 13, in certain 
embodiments either or both of non-code audio signal 
components and other noise (collectively referred to in 
this context as "noise") are used to produce a comparison 
35 value, such as a threshold, as indicated by the 
functional block 276. One or more portions of the 
encoded audio signal are compared against the comparison 
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value, as indicated by the functional block 277, to 
detect the presence of code components. Preferably, the 
encoded audio signal is first processed to isolate 
components within the frequency band or bands which may 
5 contain code components, and then these are accumulated 
over a period of time to average out noise, as indicated 
by the functional block 278. 

Referring now to Figure 14, an embodiment of an 
analog decoder in accordance with the present Invention 

10 is illustrated in block format therein. The decoder of 
Figure 14 includes an input terminal 280 which is coupled 
with four groups of component detectors 282, 284, 286 and 
288- Each group of component detectors 282 through 288 
serves to detect the presence of code components in the 

15 input audio signal representing a respective code symbol. 
In the embodiment of Figure 14, the decoder apparatus is 
arranged to detect the presence of any of 4N code 
components, where N is an integer, such that the code is 
comprised of four different symbols each represented by a 

20 unique group of N code components. Accordingly, the four 
groups 282 through 288 include 4N component detectors. 

An embodiment of one of the 4N component 
detectors of the groups 282 through 288 is illustrated in 
block format in Figure 15 and is identified therein as 

25 the component detector 290. The component detector 290 
has an input 292 coupled with the input 280 of the Figure 
14 decoder to receive the encoded audio signal. The 
component detector 290 includes an upper circuit branch 
having a noise estimate filter 294 which, in one 

30 embodiment, takes the form of a bandpass filter having a 
relatively wide passband to pass audio signal energy 
within a band centered on the frequency of the respective 
code component to be detected. In the alternative and 
preferably, the noise estimate filter 294 instead 

35 includes two filters, one of which has a passband 

extending from above the frequency of the respective code 
component to be detected and a second filter having a 
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passband with an upper edge below the frequency of the 
code component to be detected, so that together the two 
filters pass energy having frequencies above and below 
(but not including) the frequency of the component to be 
5 detected, but within a frequency neighborhood thereof. 
An output of the noise estimate filter 294 is connected 
with an input of an absolute value circuit 296 which 
produces an output signal representing the absolute value 
of the output of the noise estimate filter 294 to the 
10 input of an integrator 300 which accumulates the signals 
input thereto to produce an output value representing 
signal energy within portions of the frequency spectrum 
adjacent to but not including the frequency of the 
component to be detected and outputs this value to a non- 
15 inverting input of a difference amplifier 302 which 
operates as a logarithmic amplifier. 

The component detector of Figure 15 also 
includes a lower branch including a signal estimate 
filter 306 having an input coupled with the input 292 to 
20 receive the encoded audio signal and serving to pass a 
band of frequencies substantially narrower than the 
relatively wide band of the noise estimate filter 294 so 
that the signal estimate filter 306 passes signal 
components substantially only at the frequency of the 
25 respective code signal component to be detected. The 

signal estimate filter 306 has an output coupled with an 
input of a further absolute value circuit 308 which 
serves to produce a signal at an output thereof 
representing an absolute value the signal passed by the 
30 signal estimate filter 306. The output of the absolute 
value circuit 308 is coupled with an input of a further 
integrator 310. The integrator 310 accumulates the 
values output by the circuit 308 to produce an output 
signal representing energy within the narrow pass band of 
35 the signal estimate filter for a predetermined period of 
time. 
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Each of integrators 300 and 310 has a reset 
terminal coupled to receive a common reset signal applied 
at a terminal 312. The reset signal is supplied by a 
control circuit 314 illustrated in Figure 14 which 
5 produces the reset signal periodically. 

Returning to Figure 15, the output of the 
integrator 310 is supplied to an inverting input of the 
amplifier 302 which is operative to produce an output 
signal representing the difference between the output of 
10 the integrator 310 and that of the integrator 300. Since 
the amplifier 302 is a logarithmic amplifier, the range 
of possible output values is compressed to reduce the 
dynamic range of the output for application to a window 
comparator 316 to detect the presence or absence of a 
15 code component during a given interval as determined by 
the control circuit 314 through application of the reset 
signal. The window comparator outputs a code presence 
signal in the event that the input supplied from the 
amplifier 302 falls between a lower threshold applied as 
20 a fixed value to a lower threshold input terminal of the 
comparator 316 and a fixed upper threshold applied to an 
upper threshold input terminal of the comparator 316. 

With reference again to Figure 14, each of the 
N component detectors 290 of each component detector 
25 group couples the output of its respective window 

comparator 316 to an input of a code determination logic 
circuit 320. The circuit 320, under the control of the 
control circuit 314, accumulates the various code 
presence signals from the 4N component detector circuits 
30 290 for a multiple number of reset cycles as established 
by the control circuit 314. Upon the termination of the 
interval for detection of a given symbol, established as 
described hereinbelow, the code determination logic 
circuit 320 determines which code symbol was received as 
35 that symbol for which the greatest number of components 
were detected dturing the interval and outputs a signal 
indicating the detected code symbol at an output terminal 
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322. The output signal may be stored in memory, 
assembled into a larger message or data file, transmitted 
or otherwise utilized (for example, as a control signal) . 

5 Symbol detection intervals for the decoders 

described above in connection with Figures 11, 12 A, 12B, 
14 and 15 may be established based on the timing of 
synchronization symbols transmitted with each encoded 
message and which have a predetermined duration and 

10 order. For example, an encoded message included in an 
audio signal may be comprised of two data intervals of 
the encoded E symbol followed by two data intervals of 
the encoded S symbol, both as described above in 
connection with Figure 4. The decoders of Figures 11, 

15 12 A, 12B, 14 and 15 are operative initially to search for 
the presence of the first anticipated synchronization 
symbol, that is, the encoded E symbol which is 
transmitted during a predetermined period and determine 
its transmission interval. Thereafter, the decoders 

20 search for the presence of the code components 

characterizing the symbol S and, when it is detected, the 
decoders determine its transmission interval. From the 
detected transmission intervals, the point of transition 
from the E symbol to the S symbol is determined and, from 

25 this point, the detection intervals for each of the data 
bit symbols are set. During each detection interval, the 
decoder accumulates code components to determine the 
respective symbol transmitted diiring that interval in the 
manner described above. 

30 Although various elements of the embodiment of 

Figures 14 and 15 are implemented by analog circuits, it 
will be appreciated that the same functions carried out 
thereby may also be implemented, in whole or in part, by 
digital circuitry. 

35 With reference now to Figures 16 and 17, a 

system is illustrated therein for producing estimates of 
audiences for widely disseminated information, such as 
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television and radio programs. Figure 16 is a block 
diagram of a radio broadcasting station for broadcasting 
audio signals over the air which have been encoded to 
identify the station together with a time of broadcast. 
5 If desired, the identity of a program or segment which is 
broadcast may also be included* A program audio source 
340, such as a compact disk player, digital audio tape 
player, or live audio source is controlled by the station 
manager by means of control apparatus 342 to controllably 

10 output audio signals to be broadcast. An output 344 of 
the program audio source is coupled with an input of an 
encoder 348 in accordance with the embodiment of Figure 3 
and including the DSP 104, the bandpass filter 120^ the 
analog-to-digital converter (A/D) 124, the digital-to- 

15 analog converter (DAC) 140 and summing circuit 142 

thereof. The control apparatus 342 includes the host 
processor 90, keyboard 96 and monitor 100 of the Figure 3 
embodiment, so that the host processor included within 
the control apparatus 342 is coupled with the DSP 

20 included within the encoder 348 of Figure 16. The 

encoder 348 is operative under the control of the control 
apparatus 342 to include an encoded message periodically 
in the audio to be transmitted, the message including 
appropriate identifying data. The encoder 348 outputs 

25 the encoded audio to the input of a radio transmitter 350 
which modulates a carrier wave with the encoded program 
audio and transmits the same over the air by means of an 
antenna 352. The host processor included within the 
control apparatus 342 is programmed by means of the 

30 keyboard to control the encoder to output the appropriate 
encoded message including station identification data. 
The host processor automatically produces time of 
broadcast data by means of a reference clock circuit 
therein. 

35 Referring also to Figure 17, a personal 

monitoring device 380 of the system is enclosed by a 
housing 382 which is sufficiently small in size to be 
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carried on the person of an audience member participating 
in an audience estimate siirvey. Each of a number of 
audience members is provided with a personal monitoring 
device, such as device 380, which is to be carried on the 
5 person of the audience member dtiring specified times of 
each day dxiring a survey period, such as a predetermined 
one week period. The personal monitoring device 380 
includes an omnidirectional microphone 386 which picks up 
sounds that are available to the audience member carrying 

10 the device 380, including radio programs reproduced as 
soimd by the speaker of a radio receiver, such as the 
radio receiver 390 in Figure 17. 

The personal monitoring device 380 also 
includes signal conditioning circuitry 394 having an 

15 input coupled with an output of the microphone 386 and 
serving to amplify its output and subject the same to 
bandpass filtering both to attenuate frequencies outside 
of an audio frequency band including the various 
frequency components of the code included in the program 

20 audio by the encoder 348 of Figure 16 as well as to carry 
out anti-aliasing filtering preliminary to analog-to- 
digital conversion. 

Digital circuitry of the personal monitoring 
device 380 is illustrated in Figure 17 in functional 

25 block diagram form including a decoder block and a 
control block both of which may be implemented, for 
example, by means of a digital signal processor. A 
program and data storage memory 404 is coupled both with 
the decoder 400 to receive detected codes for storage as 

30 well as with the control block 402 for controlling the 
writing and reading operations of the memory 404. An 
input/output (I/O) circuit 406 is coupled with the memory 
404 to receive data to be output by the personal 
monitoring device 380 as well as to store information 

35 such as program instructions therein. The I/O circuit 
406 is also coupled with the control block 402 for 
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controlling input and output operations of the device 
380. 

The decoder 400 operates in accordance with the 
decoder of Figure 11 described hereinabove and outputs 
5 station identification and time code data to be stored in 
the memory 404. The personal monitoring device 380 is 
also provided with a connector, indicated schematically 
at 410, to output accumulated station identification and 
time code data stored in the memory 404 as well as to 
10 receive commands from an external device. 

The personal monitoring device 380 preferably 
is capable of operating with the docking station as 
disclosed in U.S. Patent Application Serial No. 
08/101,558 filed August 2, 1993 entitled Compliance 
15 Incentives for Audience Monitoring/Recording Devices, 
which is commonly assigned with the present application 
and which is incorporated herein by reference. In 
addition, the personal monitoring device 380 preferably 
is provided with the additional features of the portable 
broadcast exposure monitoring device which is also 
disclosed in said U.S. Patent Application Serial No. 
08/101,558. 

The docking station communicates via modem over 
telephone lines with a centralized data processing 
facility to upload the identification and time code data 
thereto to produce reports concerning audience viewing 
and/or listening. The centralized facility may also 
download information to the docking station for its use 
and/ or for provision to the device 380, such as 
executable program information. The centralized facility 
may also supply information to the docking station and/or 
device 380 over an RF channel such as an existing FM 
broadcast encoded with such information in the manner of 
the present Invention. The docking station and/or device 
380 is provided with an FM receiver (not shown for 
purposes of simplicity and clarity) which demodulates the 
encoded FM broadcast to supply the same to a decoder in 
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accordance with the present invention. The encoded FM 
broadcast can also be supplied via cable or other 
transmission medivun. 

In addition to monitoring by means of personal 
5 monitoring units, stationary units (such as set-top 

units) may be employed. The set-top units may be coupled 
to receive the encoded audio in electrical form from a 
receiver or else may employ a microphone such as 
microphone 386 of Figure 17. The set-top units may then 

10 monitor channels selected, with or without also 

monitoring audience composition, with the use of the 
present invention. 

other applications are contemplated for the 
encoding and decoding techniques of the present 

15 invention. In one application, the sound tracks of 

commercials are provided with codes for identification to 
enable commercial monitoring to ensure that commercials 
have been transmitted (by television or radio broadcast, 
or otherwise) at agreed upon times. 

20 In still other applications, control signals 

are transmitted in the form of codes produced in 
accordance with the present invention. In one such 
application, an interactive toy receives and decodes an 
encoded control signal included, in the audio portion of 

25 a television or radio broadcast or in a sound recording 
and carries out a responsive action. In another, 
parental control codes are included in audio portions of 
television or radio broadcasts or in sound recordings so 
that a receiving or reproducing device, by decoding such 

30 codes, can carry out a parental control function to 
selectively prevent reception or reproduction of 
broadcasts and recordings. Also, control codes may be 
included in cellular telephone transmissions to restrict 
unauthorized access to the use of cellular telephone 

35 ID'S. In another application, codes are included with 
telephone transmissions to distinguish voice and data 
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transmissions to appropriately control the selection of a 
transmission path to avoid corrupting transmitted data. 

Various transmitter identification functions 
may also be implemented, for example, to ensure the 
5 authenticity of military transmissions and voice 

communications with aircraft. Monitoring applications 
are also contemplated. In one such application, 
participants in market research studies wear personal 
monitors which receive coded messages added to public 

10 address or similar audio signals at retail stores or 
shopping malls to record the presence of the 
participants. In another, employees wear personal 
monitors which receive coded messages added to audio 
signals in the workplace to monitor their presence at 

15 assigned locations. 

Secure communications may also be implemented 
with the use of the encoding and decoding techniques of 
the present invention. In one such application, secure 
underwater communications are carried out by means of 

20 encoding and decoding according to the present invention 
either by assigning code component levels so that the 
codes are masked by ambient underwater sounds or by a 
sound source originating at the location of the code 
transmitter. In another, secure paging transmissions are 

25 effected by including masked codes with other over-the- 
air audio signal transmissions to be received and decoded 
by a paging device. 

The encoding and decoding techniques of the 
present invention also may be used to authenticate voice 

30 signatures. For example, in a telephone order 

application, a stored voice print may be compared with a 
live vocalization. As another example, data such as a 
security number and/ or time of day can be encoded and 
combined with a voiced utterance, and then decoded and 

35 used to automatically control processing of the voiced 
utterance. The encoding device in this scenario can be 
either an attachment to a telephone or other voice 
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communications device or else a separate fixed unit used 
when the voiced utterance is stored directly, without 
being sent over telephone lines or otherwise. A further 
application is provision of an authentication code in a 
5 memory of a portable phone, so that the voice stream 
contains the authentication code, thereby enabling 
detection of unauthorized transmissions. 

It is also possible to achieve better 
utilization of communications channel bandwidth by 

10 including data in voice or other audio transmissions. In 
one such application, data indicating readings of 
aircraft instruments are included with air-to-ground 
voice transmissions to apprise ground controllers of an 
aircraft's operational condition without the need for 

15 separate voice and data channels. Code levels are 

selected so that code components are masked by the voice 
transmissions so that interference therewith is avoided. 

Tape pirating, the unauthorized copying of 
copyrighted works such as audio/video recordings and 

20 music can also be detected by encoding a unique 

identification number on the audio portion of each 
authorized copy by means of the encoding technique of the 
present invention. If the encoded identification number 
is detected from multiple copies, unauthorized copying is 

25 then evident. 

A fiirther application determines the programs 
which have been recorded with the use of a VCR 
incorporating a decoder in accordance with the invention. 
Video programs (such as entertainment programs, 

30 commercials, etc.) are encoded according to the present 
invention with an identification code identifying the 
progreun. When the VCR is placed in a recording mode, the 
audio portions of the signals being recorded are supplied 
to the decoder to detect the identification codes 

35 therein. The detected codes are stored in a memory of 
the VCR for subsequent use in generating a report of 
recording usage. 
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Data indicating the copyrighted works which 
have been broadcast by a station or otherwise transmitted 
by a provider can be gathered with the use of the present 
invention to ascertain liability for copyright royalties. 
5 The works are encoded with respective identification 
codes which uniquely identify them, A monitoring unit 
provided with the signals broadcast or otherwise 
transmitted by one or more stations or providers provides 
audio portions thereof to a decoder according to the 

10 present invention which detects the identification codes 
present therein. The detected codes are stored in a 
memory for use in generating a report to be used to 
assess royalty liabilities. 

Proposed decoders according to the Motion 

15 Picture Experts Group (MPEG) 2 standard already include 
some elements of the acoustic expansion processing needed 
to extract encoded data according to the present 
invention, so recording inhibiting techniques (for 
example, to prevent unauthorized recording of copyrighted 

20 works) using codes according to the present invention are 
well suited for MPEG 2 decoders. An appropriate decoder 
according to the present invention is provided in the 
recorder or as an auxiliary thereto, and detects the 
presence of a copy inhibit code in audio supplied for 

25 recording. The recorder responds to the inhibit code 
thus detected to disable recording of the corresponding 
audio signal and. any accompanying signals, such as a 
video signal. Copyright information encoded according to 
the present invention is in-band, does not require 

30 additional timing or synchronization, and naturally 
accompanies the program material. 

In still further applications, programs 
transmitted over the air, cablecast or otherwise 
transmitted, or else programs recorded on tape, disk or. 

35 otherwise, include audio portions encoded with control 
signals for use by one or more viewer or listener 
operated devices. For example, a program depicting the 



wo 95^7349 



PCT/US95/03797 



67 

path a cyclist might travel includes an audio portion 
encoded according to the present invention with control 
signals for use by a stationary exercise bicycle for 
controlling pedal resistance or drag according to the 
5 apparent incline of the depicted path. As the user 
pedals the stationary bicycle, he or she views the 
program on a television or other monitor and the audio 
portion of the program is reproduced as sound • A 
microphone in the stationary bicycle transduces the 

10 reproduced sound and a decoder according to the present 
invention detects the control signals therein, providing 
the same to a pedal resistance control unit of the 
exercise bicycle. 

From the foregoing it will be appreciated that 

15 the techniques of the present invention may be 

implemented in whole or in part using analog or digital 
circuitry and that all or part of the signal processing 
functions thereof may be carried out either by hardwired 
circuits or with the use of digital signal processors, 

20 microprocessors, microcomputers, multiple processors (for 
example, parallel processors) , or the like. 

Although specific embodiments of the invention 
have been disclosed in detail herein, it is to be 
understood that the invention is not limited to those 

25 precise embodiments, and that various modifications may 
be effected therein by one skilled the art without 
departing from the scope or spirit of the invention as 
defined in the appended claims. 
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TOT IS CIAIKED ISs 

1. An apparatus for including a code having 
at least one code frequency component with an audio 
signal having a plurality of audio signal frequency 
components , compr is ing : 

first masking evaluation means for evaluating a 
masking ability of a first set of the plurality of audio 
signal frequency components to mask the at least one code 
frequency component to human hearing to produce a first 
masking evaluation; 

second masking evaluation means for evaluating 
a masking ability of a second set of the plurality of 
audio signal frequency components different from the 
first set thereof to mask the at least one code frequency 
component to human hearing to. produce a second masking 
evaluation; 

amplitude assigning means for assigning an 
amplitude to the at least one code frequency component 
based on a selected one of the first and second masking 
evaluations ; and 

code inclusion means for including the at least 
one code frequency component with the audio signal. 

2. The apparatus of claim 1, wherein the 
first set of the plurality of audio signal frequency 
components is selected from a first frequency range and 
the second set of the plurality of audio signal frequency 
components is selected from a second frequency range 
narrower than the first frequency range. 

3. The apparatus of claim 2, wherein the 
second set of the plurality of audio signal frecjuency 
components is limited substantially to a single audio 
signal frequency component. 

4. The apparatus of claim 2, wherein the 
means for including the at least one code frec[uency 
component is operative to include a plurality of code 
frequency components with the audio signal. 
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5. The apparatus of claim 4, wherein the 
plurality of code frequency components includes a first 
component and a second component having a minimum 
frequency and a maximum frequency, respectively, among 
all frequencies of the plurality of code frequency 
components and the first frequency range extends at least 
from the minimum frequency of the pltirality of code 
signal components to the maximum frecjuency thereof. 

6. The apparatus of claim 4, wherein the 
second set of the plurality of audio signal frequency 
components comprises a plurality of second sets of audio 
signal frequency components, each of the plurality of 
second sets being selected from a respective frequency 
range narrower than the first frequency range, the second 
masking evaluation means being operative to evaluate the 
ability of each of the plurality of second sets to mask 
at least a respective one of the plurality of code signal 
components to produce corresponding second masking 
evaluations, the amplitude assigning means being 
operative to assign a corresponding amplitude to each of 
the plurality of code signal components based on at least 
one of the corresponding second evaluations, the code 
inclusion means being operative to include the plurality 
of code signal components with the audio signal. 

7. The apparatus of claim 6, wherein each of 
the plurality of second sets of audio signal frequency 
components is limited substantially to a single audio 
signal frequency component. 

8. The apparatus of claim 7, wherein the 
first set of the plurality of audio signal frequency 
components is selected from a range of audio signal 
frequencies having a bandwidth corresponding to that of a 
critical band for the at least one code frequency 
component. 

9. The apparatus of claim 1, wherein the code 
comprises a plurality of code frequency component sets, 
each of the code frequency component sets representing a 
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respectively different code symbol and including a 
plurality of respectively different code frequency 
components, the code frequency components of the code 
frequency component sets forming component clusters 
spaced from one another within the frequency domain, each 
of the component clusters having a respective 
predetermined frequency range and consisting of one 
frequency component from each of the code frequency 
component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, and wherein the 
predetermined frequency range of each respective 
component cluster is smaller than the frequency amounts 
separating the respective component cluster from its 
adjacent component clusters « 

10. The apparatus of claim l, wherein the 
first masking evaluation means is operative to detect 
signal power of audio signal frequency components of the 
first set within a specified frequency range, to 
determine first and second masking factors on the 
conditions that the signal power is at each of first and 
second frequencies, respectively, within the specified 
frequency range, the second frecjuency being different 
than the first frequency, to select that one of the first 
and second masking factors which represents a smaller 
amplitude of the at least one code frequency component, 
and to determine the masking ability of the first set of 
the plurality of audio signal frequency components based 
on the selected masking factor. 

11. The apparatus of claim 1, in combination 
with means for decoding the encoded audio signal to 
detect the at least one code frec[uency component. 

12. The apparatus of claim 1, wherein the 
amplitude assigning means is operative to select said one 
of the first and second masking evaluations based on 
relative abilities of the first and second sets of the 
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pliurality of audio signal frequency components to mask 
the at least one code frequency component. 

13. A method for including a code having at 
least one code frequency component with an audio signal 
having a plurality of audio signal frequency components, 
comprising the steps of: 

evaluating a masking ability of a first set of 
the plurality of audio signal frequency components to 
mask the at least one code frequency component to human 
hearing to produce a first masking evaluation; 

evaluating a masking ability of a second set of 
the plurality of audio signal frecjuency components to 
mask the at least one code frequency component to human 
hearing to produce a second masking evaluation; 

assigning an amplitude to the at least one code 
frec^iency component based on a selected one of the first 
and second masking evaluations; and 

including the at least one code frequency 
component with the audio signal. 

14. The method of claim 13, further comprising 
the step of decoding the encoded audio signal to detect 
the at least one code frequency component. 

15. The method of claim 13, further comprising 
the step of producing the at least one code frequency 
component in response to data representing at least one 
of a broadcast source, an audio and/ or video program 
source and an audio and/or video program identification. 

16. The method of claim 13, wherein the code 
comprises a plurality of code frequency component sets, 
each of the code frequency component sets representing a 
respectively different code symbol and including a 
plurality of respectively different code frequency 
components, the code frequency components of the code 
frequency component sets forming component clusters 
spaced from one another within the frequency domain, each 
of the component clusters having a respective 
predetermined frequency range and consisting of one 
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frequency component from each of the code frequency 
component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, and wherein the 
predetermined frequency range of each respective 
component cluster is smaller than the frequency amounts 
separating the respective component cluster from its 
adjacent component clusters. 

17, The method of claim 13, wherein the step 
of evaluating the masking ability of the first set 
includes detecting signal power of audio signal frequency 
components of the first set within a specified frequency 
range, determining first and second masking factors on 
the conditions that the signal power is at each of first 
and second frequencies, respectively, within the 
specified frequency range, the second frequency being 
different than the first frequency, selecting that one of 
the first and second masking factors which represents a 
smaller amplitude of the at least one code frequency 
component, and determining the masking ability of the 
first set of the plurality of audio signal frequency 
components based on the selected masking factor, 

18. An apparatus for including a code having 
at least one code frequency component with an audio 
signal having a plurality of audio signal frequency 
components , compr is ing : 

a digital computer having an input for 
receiving the audio signal, the digital computer being 
programmed to evaluate respective masking abilities of 
first and second sets of the plurality of audio signal 
frequency components to mask the at least one code 
frequency component to human hearing to produce 
respective first and second masking evaluations, the 
second set of the plurality of audio signal frequency 
components differing from the first set thereof, the 
digital computer being further programmed to assign an 
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amplitude to the at least one code frequency component 
based on a selected one of the first and second masking 
evaluations; and 

means for including the at least one code 
frequency component with the audio signal, 

19. The apparatus of claim 18, wherein the 
digital computer is operative to select the first set of 
the plurality of audio signal frequency components as 
those of said plurality of audio signal frequency 
components within a first group of audio frequencies, and 
is further operative to select the second set of the 
plurality of audio signal frequency components from a 
second group of audio frequencies including at least one 
frequency outside the first group of audio frequencies. 

20* The apparatus of claim 18, wherein the 
digital computer includes an input for receiving data 
representing at least one of a broadcast source, an audio 
and/ or video program source and an audio and/ or video 
program identification and is programmed to produce the 
at least one code frequency component in response to said 
data. 

21. The apparatus of claim 18, in combination 
with a decoder having an input for receiving the encoded 
audio signal and operative to detect the at least one 
code frequency component. 

22. The apparatus of claim 18, wherein the 
digital computer is programmed to produce the code as a 
plurality of code frequency component sets, each of the 
code frequency component sets representing a respectively 
different code symbol and including a plurality of 
respectively different code frequency components, the 
code frequency components of the code frequency component 
sets forming component clusters spaced from one another 
within the frequency domain, each of the component 
clusters having a respective predetermined frequency 
range and consisting of one frequency component from each 
of the code frequency component sets falling within its 



wo 95/27349 



PCT/DS95/03797 



74 

respective predetermined frecjuency range, component 
clusters which are adjacent within the freG[uency domain 
being separated by respective frequency amounts, and 
wherein the predetermined frequency range of each 
respective component cluster is smaller than the 
frequency amounts separating the respective component 
cluster from its adjacent component clusters. 

23. The apparatus of claim 18, wherein the 
digital computer is programmed to detect signal power of 
audio signal frequency components of the first set within 
a specified frequency range, to determine first and 
second masking factors on the conditions that the signal 
power is at each of first and Second frequencies, 
respectively, within the specified frequency range, the 
second frequency being different than the first 
frequency, to select that one of the first and second 
masking factors which represents a. smaller amplitude of 
the at least one code . frequency component, and to assign 
the amplitude to the at least one code frequency 
component based on the selected masking factor. 

24. An apparatus for including a code having a 
plurality of code frequency components with an audio 
signal having a plurality of audio signal frequency 
components, the plurality of code frequency components 
including a first code frequency component having a first 
frequency and a second code frequency component having a 
second frequency different from the first frequency, 
comprising: 

first masking evaluation means for evaluating a 
masking ability of at least one of the plurality of audio 
signal frequency components to mask a code frequency 
component having the first frequency to human hearing to 
produce a first respective masking evaluation; 

second masking evaluation means for evaluating 
a masking ability of at least one of the plurality of 
audio signal frequency components to mask a code 
frequency component having the second frequency to human 
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hearing to produce a second respective masking 
evaluation; 

amplitude assigning means for assigning a 
respective amplitude to the first code frequency 
component based on the first respective masking 
evaluation and for assigning a respective amplitude to 
the second code frequency component based on the second 
respective masking evaluation; and 

code inclusion means for including the 
plurality of code frequency components with the audio 
signal. 

25. The apparatus of , claim 24, wherein the 
first and second respective masking evaluations comprise 
signal level data corresponding to respective levels of 
the first and second code frequency components. 

26. The apparatus of claim 24, wherein the 
code comprises a plurality of code frequency component 
sets, each of the code frequency component sets 
representing a respectively different code symbol and 
including a plurality of respectively different code 
frequency components, the code frequency components of 
the code frequency component sets forming component 
clusters spaced from one another within the frequency 
domain, each of the component clusters having a 
respective predetermined frequency range and consisting 
of one frequency component from each of the code 
frequency component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, and wherein the 
predetermined frequency range of each respective 
component cluster is smaller than the frequency amounts 
separating the respective component cluster from its 
adjacent component clusters. 

27. The apparatus of claim 24, wherein the 
first masking evaluation means is operative to detect 
signal power of audio signal frec[uency components of the 
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at least one of the plurality of audio signal frequency 
components within a specified frequency range, to 
determine first and second masking factors on the 
conditions that the signal power is at each of first and 
second frequencies, respectively, within the specified 
frequency range, the second frequency being different 
than the first frequency, to select that one of the first 
and second masking factors which represents a smaller 
amplitude of the at least one code frequency component, 
and to determine the masking ability of the at least one 
of the plurality of audio signal frequency components 
based on the selected masking factor. 

28- The apparatus of claim 24, in combination 
with means for decoding the encoded audio signal to 
detect the first and second code frequency components. 

29. The apparatus of claim 24, further 
comprising means for producing the first code frequency 
component to represent a first information symbol and for 
producing the second code frequency component to 
represent a second information symbol different from the 
first information symbol. 

30. The apparatus of claim 29, wherein the 
code inclusion means is operative to include the first 
and second code frequency components in a common interval 
of the audio signal. 

31. The apparatus of claim 24, further 
comprising means for producing the first and second code 
frequency components in response to data representing at 
least one of a broadcast source, an audio and/or video 
program source and an audio and/or video program 
identification . 

32. A method for including a code having a 
plurality of code frequency components with an audio 
signal having a plurality of audio signal frequency 
components, the plurality of code frequency components 
including a first code frequency component having a first 
frequency and a second code frequency component having a 
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second frequency different from the first frequency, 
comprising the steps of: 

evaluating a masking ability of at least one of 
the plurality of audio signal frequency components to 
mask a code frequency component having the first 
frequency to human hearing to produce a first respective 
masking evaluation; 

evaluating a masking ability of at least one of 
the plurality of audio signal frequency components to 
mask a code frequency component having the second 
frequency to human hearing to produce a second respective 
masking evaluation; 

assigning a respective amplitude to the first 
code frequency component based on the first respective 
masking evaluation and a respective amplitude to the 
second code frequency component based on the second 
respective masking evaluation; and 

including the plurality of code frequency 
components with the audio signal. 

33. The method of claim 32, further comprising 
the step of decoding the encoded audio signal to detect 
the first and second code frequency components. 

34. The method of claim 32, further comprising 
the step of producing the first and second code frequency 
components in response to data representing at least one 
of a broadcast source, an audio and/ or video program 
source and an audio and/or video program identification. 

35. The method of claim 32, wherein the code 
comprises a plurality of code frequency component sets, 
each of the code f rec[uency component sets representing a 
respectively different code symbol and including a 
plurality of respectively different code frequency 
components, the code frequency components of the code 
frequency component sets forming component clusters 
spaced from one another within the frecpiency domain, each 
of the component clusters having a respective 
predetermined frequency range and consisting of one 
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frequency component from each of the code freqniency 
component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, and wherein the 
predetermined frequency range of each respective 
component cluster is smaller than the frequency amounts 
separating the respective component cluster from its 
adjacent component clusters. 

36. The method of claim 32, wherein the step 
of evaluating the masking ability of at. least one of the 
plurality of audio signal frequency components to mask a 
code frequency component having the first frequency 
includes detecting signal power of audio signal frequency 
components within a specified frequency range, 
determining first and second masking factors on the 
conditions that the signal power is at each of first and 
second frequencies, respectively, within the specified 
frequency range, the second frequency being different 
than the first frequency, selecting that one of the first 
and second masking factors which represents a smaller 
amplitude of the at least one code frequency component^ 
and determining the masking ability of the at least one 
of the plurality of audio signal frequency components to 
mask a code frequency component having the first 
frequency based on the selected masking factor* 

37. An apparatus for including a code having a 
plurality of code frequency components with an audio 
signal having a plurality of audio signal frequency 
components, the plurality of code frequency components 
including a first code frequency component having a first 
frequency and a second code frequency component having a 
second code frequency different from the first frequency, 
comprising: 

a digital computer having an input for 
receiving the audio signal, the digital computer being 
programmed to evaluate a masking ability of at least one 
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of the plurality of audio signal frequency components to 
mask a code frequency component having the first 
frequency to human hearing to produce a first respective 
masking evaluation and to evaluate a masking ability of 
at least one of the plurality of audio signal frequency 
components to mask a code frequency component having the 
second frequency to human hearing to produce a second 
respective masking evaluation; 

the digital computer being further programmed 
to assign a corresponding amplitude to the first code 
frequency component based on the first respective masking 
evaluation and to assign a corresponding amplitude to the 
second code frequency component based on the second 
respective masking evaluation; and 

means for including the plurality of code 
frequency components with the audio signal. 

38. The apparatus of claim 37, wherein the 
first and second respective masking evaluations comprise 
signal level data corresponding to respective levels of 
the first and second code frequency components. 

39. The apparatus of claim 37, in coiobination 
with a decoder having an input for receiving the encoded 
audio signal and operative to detect the first and second 
code frequency components. 

40. The apparatus of claim 37, wherein the 
digital computer includes an input to receive data 
representing at least one of a broadcast source, an audio 
and/ or video program source, and an audio and/or video 
progrcun identification and is programmed to produce the 
first and second code freqniency components in response to 
said data. 

41. The apparatus of claim 37, wherein the 
means for including the plurality of code frequency 
components in the audio signal comprises a summing 
circuit having a first input for receiving the audio 
signal and a second input coupled with the digital 
computer to receive the plurality of code frequency 
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components and an output for providing the encoded audio 
signal. 

42. The apparatus of claim 37, wherein the 
means for including the plurality of code frequency 
components in the audio signal comprises said digital 
computer, said digital computer being programmed to add 
the plurality of code frequency components with the audio 
signal to include the plurality of code frequency 
components therewith. 

43. The apparatus of claim 37, wherein the 
digital computer is programmed to produce the code as a 
plurality of code frequency component sets, each of the 
code frequency component sets representing a respectively 
different code symbol and including a plurality of 
respectively different code frequency components, the 
code frequency components of the code frequency component 
sets forming component clusters spaced from one another 
within the frequency domain, each of the component 
clusters having a respective predetermined frequency 
range and consisting of one frequency component from each 
of the code frequency component sets falling within its 
respective predetermined frequency range, component 
clusters which are adjacent within the frequency domain 
being separated by respective frecjuency amounts, and 
wherein the predetermined frequency range of each 
respective component cluster is smaller than the 
frequency amounts separating the respective component 
cluster from its adjacent component clusters. 

44. The apparatus of claim 37, wherein the 
digital computer is programmed to evaluate the masking 
ability of the at least one of the plurality of audio 
signal frequency components by detecting signal power of 
audio signal frec[uency components within a specified 
frequency range, to detennine first and second masking 
factors with respect to the code frequency component 
having the first frequency on the conditions that the 
signal power is at each of first and second frequencies. 
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respectively, within the specified frequency range, the 
second frec[uency being different than the first 
frequency, and to select that one of the first and second 
masking factors which represents a smaller amplitude of 
the at least one code frequency component, the digital 
computer being programmed to assign the amplitude to the 
first code frequency component based on the selected 
masking factor. 

45. An apparatus for including a code having 
at least one code frequency component with an audio 
signal including a plurality of audio signal frequency 
components, comprising: 

masking evaluation means for evaluating an 
ability of at least one of the plurality of audio signal 
freG[uency components within a first audio signal interval 
on a time scale of the audio signal when reproduced as 
sound during a corresponding first time interval to mask 
the at least one code frequency component to human 
hearing when reproduced as sound during a second time 
interval corresponding to a second audio signal interval 
offset from the first audio signal interval to produce a 
first masking evaluation; 

amplitude assigning means for assigning an 
amplitude to the at least one code frequency component 
based on the first masking evaluation; and 

code inclusion means for including the at least 
one code frequency component with a portion of the audio 
signal within the second audio signal interval. 

46. The apparatus of claim 45, wherein the 
second audio signal interval follows the first audio 
signal interval on the time scale of the audio signal. 

47. The apparatus of claim 45, wherein the 
second audio signal interval precedes the first audio 
signal interval on the time scale of the audio signal. 

48. The apparatus of claim 45, in combination 
with means for decoding the encoded audio signal to 
detect the at least one code frequency component. 
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49. The apparatus of claim 45, further 
comprising means for producing the at least one code 
frequency component in response to data representing at 
least one of a broadcast soxirce, an audio and/or video 
program source and an audio and/ or video program 
ident if icat ion « 

50. A method for including a code having at 
least one code frequency component with an audio signal 
including a plurality of audio signal frequency 
components, comprising the steps of; 

evaluating an ability of at least one of the 
plurality of audio signal frequency components within a 
first audio signal interval on a time scale of the audio 
signal when reproduced as sound during a corresponding 
first time interval to mask the at least one code 
frequency component to himan hearing when reproduced as 
sound during a second time interval corresponding to a 
second audio signal interval offset from the first audio 
signal interval to produce a first masking evaluation; 

assigning an amplitude to the at least one code 
frequency component based on the first masking 
evaluation; and 

including the at least one code frequency 
component with a portion of the audio signal within the 
second audio signal interval. 

51. The method of claim 50, further comprising 
the step of decoding the encoded audio signal to detect 
the at least one code frequency component. 

52. The method of claim 50, further comprising 
the step of producing the at least one code frequency 
component in response to data representing at least one 
of a broadcast source, an audio and/ or video program 
source and an audio and/or video program identification. 

53. An apparatus for including a code having 
at least one code frequency component with an audio 
signal including a plurality of audio signal frequency 
components , compr is ing : 
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a digital computer having an input for 
receiving the audio signal, the digital computer being 
programmed to evaluate an ability of at least one of the 
plurality of audio signal frequency components within a 
first audio signal interval on a time scale of the audio 
signal when reproduced as sound during a corresponding 
first time interval to mask the at least one code 
frequency component to human hearing when reproduced as 
sound during a second time interval corresponding to a 
second audio signal interval offset from the first audio 
signal interval, to produce a first masking evaluation; 

the digital computer being further programmed 
to assign an amplitude to the a^ least one code frequency 
component based on the first masking evaluation; and 

means for including the at least one code 
frequency component with a portion of the audio signal 
within the second audio signal interval. 

54. The apparatus of claim 53, in combination 
with a decoder having an input for receiving the encoded 
audio signal and operative to detect the first and second 
code frec[uency components therein. 

55. The apparatus of claim 53, wherein the 
digital computer includes an input for receiving data 
representing at least one of a broadcast source, an audio 
and/ or video program source, and an audio and/ or video 
progrsuQ identification and is programmed to produce the 
at least one code frequency component in response to said 
data. 

56. An apparatus for including a code having 
at least one code frequency component with an audio 
signal having a plurality of audio signal frequency 
components , comprising : 

tonal signal producing means for producing a 
first tonal signal representing a first substantially 
single one of the plurality of audio signal frequency 
components ; 
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masking evaluation means for evaluating a 
masking ability of the first substantially single one of 
the plurality of audio signal frequency components to 
mask the at least one code frequency component to human 
hearing based on the first tonal signal to produce a 
first masking evaluation; 

amplitude assigning means for assigning an 
amplitude to the at least one code frequency component 
based on the first masking evaluation; and 

code inclusion means for including the at least 
one code frecpiency component with the audio signal. 

57. The apparatus of claim 56, wherein the 
tonal signal producing means is operative to produce a 
second tonal signal representing a second substantially 
single one of the plurality of audio signal frequency 
components different from the first substantially single 
one thereof, the masking evaluation means is operative to 
evaluate the ability of the second substantially single 
one of the plurality of audio signal frequency components 
to mask the at least one code frequency component to 
human hearing based on the second tonal signal to produce 
a second masking evaluation, and the amplitude assigning 
means is operative to assign an amplitude to the at least 
one code frequency component based on a selected one of 
the first and second masking evaluations. 

58. The apparatus of claim 57, wherein the 
amplitude assigning means is operative to select said one 
of the first and second masking evaluations as that one 
of the first and second masking evaluations which 
indicates a greater ability of a corresponding one of the 
first and second substantially single ones of the 
plurality of audio signal frequency components to mask 
the at least one code frequency component to human 
hearing. 

59. The apparatus of claim 56, in combination 
with decoding means for decoding the encoded audio signal 
to detect the at least one code frecpiency component. 
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60. The apparatus of claim 56, further 
comprising means for producing the at least one code 
frec[uency component in response to data representing at 
least one of a broadcast source, an audio and/or video 
program source and an audio and/ or video program 
identification • 

61. The apparatus of claim 56, wherein the 
code comprises a plurality of code frequency component 
sets, each of the code frequency component sets 
representing a respectively different code symbol and 
including a plurality of respectively different code 
frequency components, the code frequency components of 
the code frequency component sets forming component 
clusters spaced from one another within the frequency 
domain, each of the component clusters having a 
respective predetermined frequency range and consisting 
of one frequency component from each of the code 
frequency component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, and wherein the 
predetermined frequency range of each respective 
component cluster is smaller than the frequency amounts 
separating the respective component cluster from its 
adjacent component clusters. 

62. The apparatus of claim 56, wherein the 
masking evaluation means is operative to detect signal 
power of the first substantially single one of the 
plurality of the audio signal frequency components within 
a specified frequency range, to determine first and 
second masking factors on the conditions that the signal 
power is at each of first and second frequencies, 
respectively, within the specified frecpiency range, the 
second frequency being different than the first 
frequency, to select that one of the first and second 
masking factors which represents a smaller amplitude of 
the at least one code frec[uency component, and to 
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determine the masking ability of the first substantially 
single one of the plurality of the audio signal frequency 
components based on the selected masking factor. 

63. The apparatus of claim 56, wherein said 
masking evaluation means is operative to produce said 
first masking evaluation only when said at least one code 
frequency component is within a critical band of said 
first substantially single one of the plurality of audio 
signal frequency components, 

64. The apparatus of claim 56, wherein said 
code includes a plurality of code frequency components, 
and said amplitude assigning means is operative to assign 
the amplitude to the at least bne code frequency 
component based on a number of the code frequency 
components within a critical band of the at least one 
code frequency component. 

65. The apparatus of claim 56, wherein said 
tonal signal producing means is also operative to produce 
a second tonal signal representing a second substantially 
single one of the plurality of audio signal frequency 
components; said masking evaluation means is also 
operative to evaluate an ability of said second 
substantially single one of the plurality of audio signal 
frequency components to mask the at least one code 
frequency component to hiunan hearing based on the second 
tonal signal to produce a second masking evaluation; and 
said amplitude assigning means is operative to assign 
said amplitude to the at least one code frequency 
component based on the first and second masking 
evaluations . 

66. The apparatus of claim 65, wherein said 
amplitude assigning means is operative to assign the 
amplitude to the at least one code frequency component 
based on a distribution of power between said first and 
second tonal signals. 

67. A method for including a code having at 
least one code frequency component with an audio signal 
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having a plurality of audio signal frequency components, 
comprising the steps of: 

producing a first tonal signal representing a 
first substantially single one of the plurality of audio 
signal frequency components; 

evaluating a masking ability of the first 
substantially single one of the plurality of audio signal 
frequency components to mask the at least one code 
frequency component to human hearing based on the first 
tonal signal to produce a first masking evaluation; 

assigning an amplitude to the at least one code 
frequency component based on the first masking 
evaluation; and 

including the at least one code frequency 
component with the audio signal « 

68. The method of claim 67, further comprising 
the step of decoding the encoded audio signal to detect 
the at least one code frecpiency component. 

69. The method of claim 67, further comprising 
the step of producing the at least one code frequency 
component in response to data representing at least one 
of a broadcast source, an audio and/ or video program 
source and an audio and/or video program identification. 

70. The method of claim 67, wherein the code 
comprises a plurality of code frequency component sets, 
each of the code frequency component sets representing a 
respectively different code symbol and including a 
pliirality of respectively different code frequency 
components, the code frec[uency components of the code 
frequency component sets forming component clusters 
spaced from one another within the frecjuency domain, each 
of the component clusters having a respective 
predetermined frequency range and consisting of one 

f rec[uency component from each of the code frequency 
component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
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by respective frequency amounts, and wherein the 
predetermined frequency range of each respective 
component cluster is smaller than the frequency amounts 
separating the respective component cluster from its 
adjacent component clusters. 

71. The method of claim 67, wherein the step 
of evaluating the masking ability of the first 
substantially single one of the plurality of audio signal 
frequency components includes detecting signal power of 
the first substantially single one of the plurality of 
audio signal frequency components within a specified 
frequency range, determining first and second masking 
factors on the conditions that the signal power is at 
each of first and second frequencies, respectively, 
within the specified frequency^ range, the second 
frequency being different than the first frequency, 
selecting that one of the first and second masking 
factors which represents a smaller amplitude of the at 
least one code frequency component, and determining the 
masking ability of the first substantially single one of 
the plurality of audio signal frequency components based 
on the selected masking factor. 

72. The method of claim 67, wherein the step 
of evaluating a masking ability occxxrs only when said at 
least one code frequency component is within a critical 
band of said first substantially single one of the 
plurality of audio signal frequency components. 

73. The method of claim 67, wherein said code 
includes a plurality of code frequency components, and 
the step of assigning an amplitude to the at least one 
code frequency component is based on a number of the code 
frequency components within a critical band of the at 
least one code frequency component. 

74. The method of claim 67, further including 
the steps of: producing a second tonal signal 
representing a second substantially single one of the 
plurality of audio signal frequency components; 
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evaluating a masking ability of said second substantially 
single one of the plurality of audio signal frequency 
components to mask the at least one code frequency 
component to human hearing based on the second tonal 
signal to produce a second masking evaluation; and 
wherein the step of assigning assigns the amplitude to 
the at least one code frequency component based on the 
first and second masking evaluations. 

75. The method of claim 74, wherein the step 
of assigning assigns the amplitude to the at least one 
code frequency component based on a distribution of power 
between said first and second tonal signals. 

76. An apparatus for ' including a code having 
at least one code frequency component with an audio 
signal having a plurality of audio signal frequency 
components , comprising ; 

a digital computer having an input for 
receiving the audio signal, the digital computer being 
programmed to produce a first tonal signal representing a 
first substantially single one of the plurality of audio 
signal frec[uency components and to evaluate a masking 
ability of the first substantially single one of the 
plurality of audio signal frequency components to mask 
the at least one code frequency component to human 
hearing based on the first tonal signal to produce a 
first masking evaluation, the digital computer being 
further programmed to assign an amplitude to the at least 
one code frequency component based on the first masking 
evaluation; and 

code Inclusion means for including the at least 
one code frequency component with the audio signal. 

77. The apparatus of claim 76, wherein the 
digital computer includes an input for receiving data 
representing at least one of a broadcast source, an audio 
and/ or video program source and an audio and/ or video 
program identification and is programmed to produce the 
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at least one code freG[uency component in response to said 
data. 

78. The apparatus of claim 76, in combination 
with a decoder having an input for receiving the encoded 
audio signal and operative to detect the at least one 
code frequency component. 

79. The apparatus of claim 16, wherein the 
digital computer is programmed to produce the code as a 
plurality of code frecjuency component sets, each of the 
code frequency component sets representing a respectively 
different code symbol and including a plurality of 
respectively different code frequency components, the 
code frequency components of the code frequency component 
sets forming component clusters spaced from one another 
within the frequency domain, each of the component 
clusters having a respective predetermined frequency 
range and consisting of one frequency component from each 
of the code frequency component sets falling within its 
respective predetermined frequency range, component 
clusters which are adjacent within the frequency domain 
being separated by respective frequency amounts, and 
wherein the predetermined frequency range of each 
respective component cluster is smaller than the 
frequency amounts separating the respective component 
cluster from its adjacent component clusters. 

80. The apparatus of claim 76, wherein the 
digital computer is programmed to detect signal power of 
the first substantially single one of the plurality of 
the audio signal frequency components within a specified 
frequency range, to determine first and second masking 
factors on the conditions that the signal power is at 
each of first and second frecjuencies, respectively, 
within the specified frec[uency range, the second 
frequency being different than the first frecpiency, and 
to select that one of the first and second masking 
factors which represents a smaller amplitude of the at 
least one code frequency component, the digital computer 
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being further programmed to assign the amplitude to the 
at least one code frequency component based on the 
selected masking factor. 

81. The apparatus of claim 76, wherein the 
digital computer is programmed to produce said first 
masking evaluation only when said at least one code 
frequency component is within a critical band of said 
first substantially single one of the plurality of audio 
signal frequency components. 

82. The apparatus of claim 76, wherein said 
code includes a plurality of code frequency components, 
and said digital computer is programmed to assign the 
amplitude to the at least one code frequency component 
based on a number of the code frequency components within 
a critical band of the at least one code frequency 
component • 

83. The apparatus of claim 76, wherein said 
digital computer is programmed to produce a second tonal 
signal representing a second substantially single one of 
the plurality of audio signal frequency components; to 
evaluate an ability of said second substantially single 
one of the plurality of audio signal frequency components 
to mask the at least one code frequency component to 
human hearing based on the second tonal signal to produce 
a second masking evaluation; and to assign said amplitude 
to the at least one code frequency component based on the 
first and second masking evaluations. 

84. The apparatus of claim 83, wherein said 
digital .computer is programmed to assign the amplitude to 
the at least one code frequency component based on a 
distribution of power between said first and second tonal 
signals. 

85. An apparatus for encoding an audio signal, 
comprising: 

means for generating a code comprising a 
pliirality of code frequency component sets, each of the 
code frecpiency component sets representing a respectively 
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different code symbol and including a plurality of 
respectively different code frequency components, the 
code frecjuency components of the code frequency component 
sets forming component clusters spaced from one another 
within the frequency domain, each of the component 
clusters having a respective predetermined frequency 
range and consisting of one frequency component from each 
of the code frequency component sets falling within its 
respective predetermined frequency range, component 
clusters which are adjacent within the frequency domain 
being separated by respective frequency, amounts, the 
predetermined frequency range of each respective 
component cluster being smaller than the frequency 
amounts separating the respective component cluster from 
its adjacent component clusters; and 

code inclusion means for combining the code 
with the audio signal* 

86. A method for encoding an audio signal, 
comprising: 

generating a code comprising a plurality of 
code frequency component sets, each of the code frequency 
component sets representing a respectively different code 
symbol and including a plurality of respectively 
different code frequency components, the code frequency 
components of the code frequency component sets forming 
component clusters spaced from one another within the 
frequency domain, each of the component clusters having a 
respective predetermined frequency range and consisting 
of one frec[uency component from each of the code 
frequency component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, the predetermined 
frequency range of each respective component cluster 
being smaller than the frequency amounts separating the 
respective component cluster from its adjacent component 
clusters ; and 
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combining the code with the audio signal. 

87. An apparatus for encoding an audio signal, 
comprising: 

a digital computer having an input for 
receiving the audio signal, the digital computer being 
programmed to produce a code comprising a plurality of 
code frequency component sets, each of the code frequency 
component sets representing a respectively different code 
symbol and including a plurality of respectively 
different code frequency components, the code frequency 
components of the code frequency component sets forming 
component clusters spaced from one another within the 
frequency domain, each of the component clusters having a 
respective predetermined frequency range and consisting 
of one frequency component from each of the code 
frequency component sets falling within its respective 
predetermined frequency range, component clusters which 
are adjacent within the frequency domain being separated 
by respective frequency amounts, the predetermined 
frequency range of each respective component cluster 
being smaller than the freG[uency amounts separating the 
respective component cluster from its adjacent component 
clusters; and 

means for combining the code with the audio 

signal. 

88. An apparatus for detecting a code in an 
encoded audio signal, the encoded audio signal including 
a plurality of audio frequency signal components and at 
least one code frequency component having an amplitude 
and an audio frequency selected for masking the code 
frequency component to human hearing by at least one of 
the plurality of audio frequency signal components, 
comprising: 

means for establishing an expected code 
amplitude of the at least one code frequency component 
based on the encoded audio signal; and 
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component in the encoded audio signal based on the 
expected code amplitude. 

89. The apparatus of claim 88, further 
comprising means for detecting a first component of the 
encoded audio signal at the audio frequency of the at 
least one code frequency component, and wherein the means 
for detecting the code frequency component is operative 
to determine whether an amplitude of the detected first 
component corresponds with the expected code amplitude. 

90. The apparatus of claim 88, wherein the 
means for detecting the first component of the encoded 
audio signal comprises means for separating the encoded 
audio signal into frequency component groups each 
including one or more components within a corresponding 
frequency range, a first one of the frequency component 
groups having a corresponding frequency range including 
the audio frequency of the at least one code frequency 
component. 

91. A method for detecting a code in an 
encoded audio signal, the encoded audio signal including 
a plurality of audio frequency signal components and at 
least one code frequency component having an amplitude 
and an audio frequency selected for masking the code 
frequency component to human hearing by at least one of 
the plurality of audio frec[uency signal components, 
comprising the steps of: 

establishing an expected code amplitude of the 
at least one code freqniency component based on the 
encoded audio signal; and 

detecting the code frequency component in the 
encoded audio signal based on the expected code 
amplitude. 

92. A programmed digital computer for 
detecting a code in an encoded audio signal, the encoded 
audio signal including a plurality of audio frequency 
signal components and at least one code frequency 



wo 95/27349 



PCT/US95/03797 



component having an amplitude and an audio frequency 
selected for masking the code frequency component to 
human hearing by at least one of the plurality of audio 
frequency signal components, comprising: 

an input for receiving the encoded audio 
signal; a processor programmed to establish an expected 
code amplitude of the at least one code frequency 
component based on the encoded audio signal, to detect 
the code frequency component in the encoded audio signal 
based on the expected code amplitude and to produce a 
detected code output signal based on the detected code 
frec[uency component; and 

an output coupled with the processor for 
providing the detected code output signal, 

93, An apparatus for detecting a code in an 
encoded audio signal, the encoded audio signal having a 
plurality of frequency components including a plurality 
of audio frequency signal components and at least one 
code frequency componeftt having a predetermined audio 
frequency and a predetermined amplitude for 
distinguishing the at least one code f req;uency component 
from the plurality of audio frequency signal components, 
comprising: 

means for determining an amplitude of frequency 
components of the encoded audio signal within a first 
range of audio freq[uencies including the predetermined 
audio frequency of the at least one code frequency 
component; 

means for establishing a noise amplitude for 
the first range of audio frequencies; and 

means for detecting the presence of the at 
least one code frequency component in the first range of 
audio frequencies based on the established noise 
amplitude thereof and the determined amplitude of 
frequency components therein. 

94. A method for detecting a code in an 
encoded audio signal, the encoded audio signal having a 
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plxurality of frecjuency components Including a plurality 
of audio frequency signal components and at least one 
code frequency component having a predetermined audio 
frec^^ency and a predetermined amplitude for 
distinguishing the at least one code frequency component 
from the plurality of audio frequency signal components, 
comprising the steps of: 

determining an simplitude of a frequency 
component of the encoded audio signal within a first 
range of audio frecpiencies including the predetermined 
audio frequency of the at least one code frequency 
component ; 

establishing a noise' amplitude for the first 
range of audio frequencies; and 

detecting the presence of the at least one code 
frequency component in the first range of audio 
frequencies based on the established noise amplitude 
thereof and the determined amplitude of the freG[uency 
component therein. 

95. A digital computer for detecting a code in 
an encoded audio signal, the encoded audio signal having 
a plurality of frequency components including a plurality 
of audio frequency signal components and at least one 
code frequency component having a predetermined audio 
frequency and a predetermined amplitude for 
distinguishing the at least one code frequency component 
from the plurality of audio frequency signal components, 
comprising: an input for receiving the encoded audio 
signal; a processor coupled with the input to receive the 
encoded audio signal and programmed to determine an 
amplitude of a frequency component of the encoded audio 
signal within a first range of audio frequencies 
including the predetermined audio frequency of the at 
least one code frequency component; the processor being 
further programmed to establish a noise amplitude for the 
first range of audio frequencies and to detect the 
presence of the at least one code frequency component in 
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the first range of audio frequencies based on the 
established noise amplitude thereof and the determined 
amplitude of the frequency component therein; the 
processor being operative to produce a code output signal 
based on the detected presence of the at least one code 
frequency component; and an output terminal coupled with 
the processor to provide the code signal thereat. 
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